[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:15 dist-test-slave-10pc krb5kdc[18724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:15 dist-test-slave-10pc krb5kdc[18724](info): set up 2 sockets
Aug 09 07:48:15 dist-test-slave-10pc krb5kdc[18724](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 07:48:16.454654 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:16 dist-test-slave-10pc krb5kdc[18724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:16.589819 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45225
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:45225
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:16.718971 18737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:16.719267 18737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:16.719357 18737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:16.723418 18737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:16.723500 18737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:16.723539 18737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:16.723595 18737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:16.723634 18737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:16.729103 18737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:45225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:16.730517 18737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:16.731853 18737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:16.743785 18743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:16.745160 18742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:16.745301 18745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:16.746366 18737 server_base.cc:1034] running on GCE node
I20260809 07:48:16.747121 18737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:16.748852 18737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:16.750037 18737 hybrid_clock.cc:648] HybridClock initialized: now 1786261696749993 us; error 81 us; skew 500 ppm
Aug 09 07:48:16 dist-test-slave-10pc krb5kdc[18724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261696, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:16.753082 18737 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:16.754209 18737 webserver.cc:460] Webserver started at http://127.18.71.126:44249/ using document root <none> and password file <none>
I20260809 07:48:16.755152 18737 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:16.755255 18737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:16.755658 18737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:16.758338 18737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "091f67c0aeae4013b465f6dccb91cea2"
format_stamp: "Formatted at 2026-08-09 07:48:16 on dist-test-slave-10pc"
server_key: "48788e544faacd566ae85822350176fa"
server_key_iv: "1ceade8e73babe9bb2746437ec16423b"
server_key_version: "encryptionkey@0"
I20260809 07:48:16.758948 18737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "091f67c0aeae4013b465f6dccb91cea2"
format_stamp: "Formatted at 2026-08-09 07:48:16 on dist-test-slave-10pc"
server_key: "48788e544faacd566ae85822350176fa"
server_key_iv: "1ceade8e73babe9bb2746437ec16423b"
server_key_version: "encryptionkey@0"
I20260809 07:48:16.764074 18737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:48:16.767863 18752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:16.769510 18737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:16.769711 18737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "091f67c0aeae4013b465f6dccb91cea2"
format_stamp: "Formatted at 2026-08-09 07:48:16 on dist-test-slave-10pc"
server_key: "48788e544faacd566ae85822350176fa"
server_key_iv: "1ceade8e73babe9bb2746437ec16423b"
server_key_version: "encryptionkey@0"
I20260809 07:48:16.769830 18737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:16.782950 18737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:16.786666 18737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:16.786911 18737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:16.799804 18737 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:45225
I20260809 07:48:16.799930 18806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:45225 every 8 connection(s)
I20260809 07:48:16.801455 18737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:16.806806 18807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:16.808286 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 18737
I20260809 07:48:16.808552 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:16.810818 18717 external_mini_cluster.cc:1428] Setting key 6252a47e6580e77c40c272081f2b5cd0
I20260809 07:48:16.819579 18807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2: Bootstrap starting.
I20260809 07:48:16.825569 18807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:16.827865 18807 log.cc:824] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:16.831370 18807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2: No bootstrap required, opened a new log
Aug 09 07:48:16 dist-test-slave-10pc krb5kdc[18724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:16.839520 18807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091f67c0aeae4013b465f6dccb91cea2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45225 } }
I20260809 07:48:16.839859 18807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:16.839965 18807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 091f67c0aeae4013b465f6dccb91cea2, State: Initialized, Role: FOLLOWER
I20260809 07:48:16.840971 18807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091f67c0aeae4013b465f6dccb91cea2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45225 } }
I20260809 07:48:16.841207 18807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:16.841344 18807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:16.841495 18807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:16.842581 18807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091f67c0aeae4013b465f6dccb91cea2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45225 } }
I20260809 07:48:16.843125 18807 leader_election.cc:302] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 091f67c0aeae4013b465f6dccb91cea2; no voters:
I20260809 07:48:16.843541 18807 leader_election.cc:288] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:16.843786 18812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:16.844189 18812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [term 1 LEADER]: Becoming Leader. State: Replica: 091f67c0aeae4013b465f6dccb91cea2, State: Running, Role: LEADER
I20260809 07:48:16.844657 18812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091f67c0aeae4013b465f6dccb91cea2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45225 } }
I20260809 07:48:16.845386 18807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:16.846586 18812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 091f67c0aeae4013b465f6dccb91cea2. Latest consensus state: current_term: 1 leader_uuid: "091f67c0aeae4013b465f6dccb91cea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091f67c0aeae4013b465f6dccb91cea2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45225 } } }
I20260809 07:48:16.846719 18812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:16.849289 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:16.818693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57420 (local address 127.18.71.126:45225)
0809 07:48:16.819226 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:16.819242 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:16.819369 (+ 127us) server_negotiation.cc:407] Connection header received
0809 07:48:16.825413 (+ 6044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:16.825431 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:16.825928 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:16.826222 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:16.830495 (+ 4273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:16.831606 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:16.833871 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:16.834507 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:16.834653 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:16.841054 (+ 6401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:16.841090 (+ 36us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:16.841102 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:16.841130 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:16.843491 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:16.844243 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:16.844248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:16.844254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:16.844421 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:16.844918 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:16.844922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:16.844923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:16.845680 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:16.845927 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:16.846733 (+ 806us) server_negotiation.cc:299] Negotiation successful
0809 07:48:16.847133 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":194,"threads_started":1}
I20260809 07:48:16.851979 18814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "091f67c0aeae4013b465f6dccb91cea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "091f67c0aeae4013b465f6dccb91cea2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45225 } } }
I20260809 07:48:16.852115 18814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:16.852844 18825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:16.857570 18825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:16.865483 18825 catalog_manager.cc:1347] Generated new cluster ID: 301235e1bdd0492aa4159c1d5b92a927
I20260809 07:48:16.865558 18825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:16.880863 18825 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:16.881896 18825 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:16.892905 18825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 091f67c0aeae4013b465f6dccb91cea2: Generated new TSK 0
I20260809 07:48:16.894071 18825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:16.935149 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45225
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:17.099864 18836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:17.100168 18836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:17.100215 18836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:17.104575 18836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:17.104643 18836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:17.104705 18836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:17.110320 18836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:17.111451 18836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:17.112545 18836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:17.124318 18842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:17.124418 18836 server_base.cc:1034] running on GCE node
W20260809 07:48:17.124305 18841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:17.124279 18845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:17.124977 18836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:17.125634 18836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:17.126806 18836 hybrid_clock.cc:648] HybridClock initialized: now 1786261697126785 us; error 35 us; skew 500 ppm
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:17.130200 18836 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:17.131136 18836 webserver.cc:460] Webserver started at http://127.18.71.65:42421/ using document root <none> and password file <none>
I20260809 07:48:17.131709 18836 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:17.131819 18836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:17.132081 18836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:17.134591 18836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "042fc813d57643f2ae8cc0f9aa4d20b7"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "3094116f1bf35624971217000ea7b976"
server_key_iv: "6108b0e27a03aae257faf13afa01f072"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.135257 18836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "042fc813d57643f2ae8cc0f9aa4d20b7"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "3094116f1bf35624971217000ea7b976"
server_key_iv: "6108b0e27a03aae257faf13afa01f072"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.139639 18836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:48:17.142359 18852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.143628 18836 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:17.143778 18836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "042fc813d57643f2ae8cc0f9aa4d20b7"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "3094116f1bf35624971217000ea7b976"
server_key_iv: "6108b0e27a03aae257faf13afa01f072"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.143898 18836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:17.155465 18836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:17.159066 18836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:17.159308 18836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:17.160084 18836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:17.161345 18836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:17.161422 18836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.161497 18836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:17.161546 18836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.181185 18836 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:40511
I20260809 07:48:17.181205 18972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:40511 every 8 connection(s)
I20260809 07:48:17.182189 18836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:17.183883 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 18836
I20260809 07:48:17.183995 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:17.184309 18717 external_mini_cluster.cc:1428] Setting key 1abe3b4531d97c0ebd383d2a248d935c
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:17.198637 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.184701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:44877 (local address 127.18.71.126:45225)
0809 07:48:17.184839 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:17.184842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:17.186389 (+ 1547us) server_negotiation.cc:407] Connection header received
0809 07:48:17.187718 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:17.187721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:17.187765 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:17.187835 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:17.189326 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.190252 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.191453 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.191945 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.192076 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.194638 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:17.194642 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:17.194647 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:17.194675 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:17.196425 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.196964 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.196967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.196969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.197056 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.197405 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.197408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.197410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.197604 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:17.197714 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:17.198301 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 07:48:17.198445 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:48:17.199501 18975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.185226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45225 (local address 127.18.71.65:44877)
0809 07:48:17.186205 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:17.186261 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:17.187479 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:17.188023 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:17.188035 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:17.188602 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:17.189136 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:17.189151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.190397 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.190400 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:17.191314 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:17.191323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.192138 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.192142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:17.192284 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.192970 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:17.192993 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:17.194464 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:17.196574 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:17.196579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:17.196591 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:17.196838 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:17.197156 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:17.197160 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:17.197162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:17.197302 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:17.197728 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:17.197737 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:17.198060 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 07:48:17.198258 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 07:48:17.198591 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":230,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
I20260809 07:48:17.200897 18973 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45225
I20260809 07:48:17.201157 18973 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:17.201666 18973 heartbeater.cc:511] Master 127.18.71.126:45225 requested a full tablet report, sending...
I20260809 07:48:17.203689 18769 ts_manager.cc:194] Registered new tserver with Master: 042fc813d57643f2ae8cc0f9aa4d20b7 (127.18.71.65:40511)
I20260809 07:48:17.205250 18769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:44877
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:17.216131 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45225
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:17.342813 18980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:17.343115 18980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:17.343187 18980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:17.347594 18980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:17.347679 18980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:17.347772 18980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:17.353502 18980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:17.354594 18980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:17.355674 18980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:17.365867 18986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:17.365902 18988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:17.366133 18980 server_base.cc:1034] running on GCE node
W20260809 07:48:17.365875 18985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:17.366575 18980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:17.367223 18980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:17.368389 18980 hybrid_clock.cc:648] HybridClock initialized: now 1786261697368353 us; error 46 us; skew 500 ppm
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:17.371205 18980 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:17.372014 18980 webserver.cc:460] Webserver started at http://127.18.71.66:33387/ using document root <none> and password file <none>
I20260809 07:48:17.372596 18980 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:17.372663 18980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:17.372828 18980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:17.374964 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "c8393edc938a4ef0a205d7fac627eb46"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "0655c5a0e0ea3d198dc8d7a53e4bbe27"
server_key_iv: "e404156cb3d1eab502d4523a827579b5"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.375496 18980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "c8393edc938a4ef0a205d7fac627eb46"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "0655c5a0e0ea3d198dc8d7a53e4bbe27"
server_key_iv: "e404156cb3d1eab502d4523a827579b5"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.379120 18980 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 07:48:17.381685 18995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.383046 18980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:17.383174 18980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "c8393edc938a4ef0a205d7fac627eb46"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "0655c5a0e0ea3d198dc8d7a53e4bbe27"
server_key_iv: "e404156cb3d1eab502d4523a827579b5"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.383281 18980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:17.407887 18980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:17.410787 18980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:17.411017 18980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:17.411664 18980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:17.412566 18980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:17.412616 18980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.412676 18980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:17.412709 18980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.421515 18980 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:39833
I20260809 07:48:17.421551 19119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:39833 every 8 connection(s)
I20260809 07:48:17.422408 18980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:17.423087 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 18980
I20260809 07:48:17.423204 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:17.423485 18717 external_mini_cluster.cc:1428] Setting key 2c7fef8acac01733a7e2fd8f1461940d
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:48:17.438318 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.424340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:33625 (local address 127.18.71.126:45225)
0809 07:48:17.424505 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:17.424510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:17.425607 (+ 1097us) server_negotiation.cc:407] Connection header received
0809 07:48:17.427739 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:17.427744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:17.427806 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:17.427903 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:17.429296 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.429823 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.431712 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.431927 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.432007 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.434922 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:17.434925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:17.434927 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:17.434948 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:17.436260 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.436831 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.436834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.436835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.436908 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.437253 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.437256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.437258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.437429 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:17.437536 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:17.438021 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 07:48:17.438149 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:48:17.439062 19122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.424851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45225 (local address 127.18.71.66:33625)
0809 07:48:17.425416 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:17.425475 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:17.427509 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:17.428058 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:17.428064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:17.428413 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:17.429133 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:17.429157 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.429957 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.429964 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:17.431583 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:17.431599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.432091 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.432097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:17.432342 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.432981 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:17.433002 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:17.434752 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:17.436386 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:17.436392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:17.436406 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:17.436716 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:17.437001 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:17.437005 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:17.437007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:17.437149 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:17.437543 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:17.437552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:17.437788 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 07:48:17.437964 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 07:48:17.438270 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":151,"threads_started":1}
I20260809 07:48:17.440294 19120 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45225
I20260809 07:48:17.440598 19120 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:17.441087 19120 heartbeater.cc:511] Master 127.18.71.126:45225 requested a full tablet report, sending...
I20260809 07:48:17.442199 18769 ts_manager.cc:194] Registered new tserver with Master: c8393edc938a4ef0a205d7fac627eb46 (127.18.71.66:39833)
I20260809 07:48:17.443084 18769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:33625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:17.455718 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45225
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:17.573446 19125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:17.573725 19125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:17.573793 19125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:17.577872 19125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:17.577927 19125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:17.578015 19125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:17.583248 19125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45225
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:17.584631 19125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:17.585819 19125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:17.595469 19130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:17.595595 19133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:17.595474 19131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:17.595945 19125 server_base.cc:1034] running on GCE node
I20260809 07:48:17.596614 19125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:17.597486 19125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:17.598671 19125 hybrid_clock.cc:648] HybridClock initialized: now 1786261697598651 us; error 26 us; skew 500 ppm
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:17.602372 19125 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:17.603379 19125 webserver.cc:460] Webserver started at http://127.18.71.67:40119/ using document root <none> and password file <none>
I20260809 07:48:17.603941 19125 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:17.604044 19125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:17.604269 19125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:17.606389 19125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "8c8dfcb2141d47eebdfbd86131b2f36f"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "5715e39fd296c7a88387d3a0d4661cb7"
server_key_iv: "320ac22404f026e19888bd2e999dfb83"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.606956 19125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "8c8dfcb2141d47eebdfbd86131b2f36f"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "5715e39fd296c7a88387d3a0d4661cb7"
server_key_iv: "320ac22404f026e19888bd2e999dfb83"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.610855 19125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:48:17.613524 19141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.614837 19125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:17.614928 19125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "8c8dfcb2141d47eebdfbd86131b2f36f"
format_stamp: "Formatted at 2026-08-09 07:48:17 on dist-test-slave-10pc"
server_key: "5715e39fd296c7a88387d3a0d4661cb7"
server_key_iv: "320ac22404f026e19888bd2e999dfb83"
server_key_version: "encryptionkey@0"
I20260809 07:48:17.615061 19125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:17.632342 19125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:17.635620 19125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:17.635821 19125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:17.636446 19125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:17.637305 19125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:17.637352 19125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.637414 19125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:17.637450 19125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:17.648298 19125 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:37191
I20260809 07:48:17.648411 19259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:37191 every 8 connection(s)
I20260809 07:48:17.649413 19125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:17.654978 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 19125
I20260809 07:48:17.655129 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:17.655462 18717 external_mini_cluster.cc:1428] Setting key 7d3fc9b5f8bced82a9adf98afe4c369d
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:17.664739 18810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.651356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:48953 (local address 127.18.71.126:45225)
0809 07:48:17.651526 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:17.651533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:17.652620 (+ 1087us) server_negotiation.cc:407] Connection header received
0809 07:48:17.653513 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:17.653519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:17.653588 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:17.653700 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:17.654898 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.655686 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.657379 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.657735 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.657860 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.661000 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:17.661004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:17.661007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:17.661048 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:17.662635 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.663175 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.663178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.663180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.663268 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.663635 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.663638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.663640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.663827 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:17.663937 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:17.664429 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 07:48:17.664579 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:48:17.665627 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.651704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45225 (local address 127.18.71.67:48953)
0809 07:48:17.652424 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:17.652506 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:17.653307 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:17.653866 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:17.653872 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:17.654242 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:17.654736 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:17.654750 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.655848 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.655852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:17.657250 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:17.657263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.657923 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.657930 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:17.658194 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.658900 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:17.658929 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:17.660827 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:17.662770 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:17.662776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:17.662791 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:17.663061 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:17.663367 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:17.663371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:17.663373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:17.663513 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:17.663937 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:17.663945 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:17.664189 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 07:48:17.664372 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 07:48:17.664745 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":139,"threads_started":1}
I20260809 07:48:17.666818 19260 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45225
I20260809 07:48:17.667078 19260 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:17.667587 19260 heartbeater.cc:511] Master 127.18.71.126:45225 requested a full tablet report, sending...
I20260809 07:48:17.668720 18769 ts_manager.cc:194] Registered new tserver with Master: 8c8dfcb2141d47eebdfbd86131b2f36f (127.18.71.67:37191)
I20260809 07:48:17.669679 18769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:48953
I20260809 07:48:17.670549 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:17.704198 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.691611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.18.71.65:40511)
0809 07:48:17.691894 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:17.691904 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:17.691940 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:48:17.692133 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:17.692139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:17.692331 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:17.692504 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:17.693538 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.694314 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.695811 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.696203 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.696289 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.699458 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:17.699472 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:17.699494 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:17.699531 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:17.701730 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.702204 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.702213 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.702217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.702348 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.702638 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.702643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.702646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.703137 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:17.703475 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:17.703651 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 07:48:17.703910 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":115,"threads_started":1}
W20260809 07:48:17.705395 18882 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60950
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[18724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:17.732366 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:17.724342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.18.71.65:40511)
0809 07:48:17.724536 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:17.724541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:17.724708 (+ 167us) server_negotiation.cc:407] Connection header received
0809 07:48:17.724798 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:17.724800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:17.724846 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:17.724910 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:17.725573 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.726107 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.726903 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:17.727126 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:17.727200 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:17.729453 (+ 2253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:17.729456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:17.729458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:17.729480 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:17.730895 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.731278 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.731281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.731282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.731356 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:17.731598 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:17.731600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:17.731601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:17.731751 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:17.731849 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:17.732054 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 07:48:17.732152 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 07:48:17.733506 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 18836
I20260809 07:48:17.743124 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 18980
I20260809 07:48:17.754319 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 19125
I20260809 07:48:17.761068 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 18737
2026-08-09T07:48:17Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2415 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[19274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[19274](info): set up 2 sockets
Aug 09 07:48:17 dist-test-slave-10pc krb5kdc[19274](info): commencing operation
krb5kdc: starting...
W20260809 07:48:18.836601 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:18 dist-test-slave-10pc krb5kdc[19274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:18.963229 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45283
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:45283
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:19.074963 19287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:19.075258 19287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:19.075353 19287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:19.079295 19287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:19.079376 19287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:19.079416 19287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:19.079448 19287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:19.079480 19287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:19.084637 19287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:45283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:19.085695 19287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:19.086710 19287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:19.095286 19294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:19.095263 19295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:19.095427 19287 server_base.cc:1034] running on GCE node
W20260809 07:48:19.095289 19297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:19.096088 19287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:19.097302 19287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:19.098502 19287 hybrid_clock.cc:648] HybridClock initialized: now 1786261699098460 us; error 39 us; skew 500 ppm
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261699, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:19.101215 19287 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:19.102073 19287 webserver.cc:460] Webserver started at http://127.18.71.126:38189/ using document root <none> and password file <none>
I20260809 07:48:19.102626 19287 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:19.102716 19287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:19.102954 19287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:19.105048 19287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "baf055d0a3464737a7d6a926f9a8aec2"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "4456fef29b7720b31b8a6f65a84eba22"
server_key_iv: "eceef9a256263c3437b4cc49d1d675af"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.105634 19287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "baf055d0a3464737a7d6a926f9a8aec2"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "4456fef29b7720b31b8a6f65a84eba22"
server_key_iv: "eceef9a256263c3437b4cc49d1d675af"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.109318 19287 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:48:19.111768 19304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.113045 19287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:48:19.113188 19287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "baf055d0a3464737a7d6a926f9a8aec2"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "4456fef29b7720b31b8a6f65a84eba22"
server_key_iv: "eceef9a256263c3437b4cc49d1d675af"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.113279 19287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:19.122709 19287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:19.125798 19287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:19.125990 19287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:19.132932 19287 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:45283
I20260809 07:48:19.132977 19360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:45283 every 8 connection(s)
I20260809 07:48:19.133895 19287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:19.136739 19361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:19.139256 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 19287
I20260809 07:48:19.139371 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:19.139659 18717 external_mini_cluster.cc:1428] Setting key 6e7cd4d8b15d0a9931a0454f82649008
I20260809 07:48:19.142997 19361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2: Bootstrap starting.
I20260809 07:48:19.145283 19361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:19.146032 19361 log.cc:824] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:19.147876 19361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2: No bootstrap required, opened a new log
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:19.152100 19361 raft_consensus.cc:348] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf055d0a3464737a7d6a926f9a8aec2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45283 } }
I20260809 07:48:19.152356 19361 raft_consensus.cc:374] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:19.152415 19361 raft_consensus.cc:729] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baf055d0a3464737a7d6a926f9a8aec2, State: Initialized, Role: FOLLOWER
I20260809 07:48:19.152931 19361 consensus_queue.cc:260] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf055d0a3464737a7d6a926f9a8aec2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45283 } }
I20260809 07:48:19.153110 19361 raft_consensus.cc:388] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:19.153173 19361 raft_consensus.cc:482] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:19.153270 19361 raft_consensus.cc:3037] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:19.153980 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.141112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.18.71.126:45283)
0809 07:48:19.141548 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.141562 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.141597 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:48:19.142229 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.142245 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.142555 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.142900 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:19.143703 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.144765 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.145707 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.146216 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.146384 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.148931 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:19.148944 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:19.148957 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:19.148982 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:19.150903 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.151366 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.151374 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.151385 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.151494 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.151784 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.151787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.151789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.152173 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:19.152388 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.152688 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.152991 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":138,"threads_started":1}
I20260809 07:48:19.154462 19361 raft_consensus.cc:504] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf055d0a3464737a7d6a926f9a8aec2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45283 } }
I20260809 07:48:19.154807 19361 leader_election.cc:302] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: baf055d0a3464737a7d6a926f9a8aec2; no voters:
I20260809 07:48:19.155161 19361 leader_election.cc:288] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:19.155236 19367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:19.155437 19367 raft_consensus.cc:686] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [term 1 LEADER]: Becoming Leader. State: Replica: baf055d0a3464737a7d6a926f9a8aec2, State: Running, Role: LEADER
I20260809 07:48:19.155761 19367 consensus_queue.cc:237] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf055d0a3464737a7d6a926f9a8aec2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45283 } }
I20260809 07:48:19.156529 19361 sys_catalog.cc:564] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:19.157016 19367 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader baf055d0a3464737a7d6a926f9a8aec2. Latest consensus state: current_term: 1 leader_uuid: "baf055d0a3464737a7d6a926f9a8aec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf055d0a3464737a7d6a926f9a8aec2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45283 } } }
I20260809 07:48:19.157131 19367 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:19.157467 19368 sys_catalog.cc:455] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "baf055d0a3464737a7d6a926f9a8aec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "baf055d0a3464737a7d6a926f9a8aec2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45283 } } }
I20260809 07:48:19.157575 19368 sys_catalog.cc:458] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:19.157660 19372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:19.161082 19372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:19.166872 19372 catalog_manager.cc:1347] Generated new cluster ID: 01560fb5edf340d5a72e2cee8b8a0bdc
I20260809 07:48:19.166934 19372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:19.189055 19372 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:19.190045 19372 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:19.194998 19372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P baf055d0a3464737a7d6a926f9a8aec2: Generated new TSK 0
I20260809 07:48:19.195844 19372 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:19.236552 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45283
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:19.349507 19389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:19.349784 19389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:19.349859 19389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:19.353900 19389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:19.353983 19389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:19.354070 19389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:19.359548 19389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:19.360582 19389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:19.361603 19389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:19.371412 19397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:19.371440 19395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:19.371441 19394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:19.371632 19389 server_base.cc:1034] running on GCE node
I20260809 07:48:19.372115 19389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:19.372929 19389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:19.374259 19389 hybrid_clock.cc:648] HybridClock initialized: now 1786261699374251 us; error 41 us; skew 500 ppm
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261699, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:19.377216 19389 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:19.378199 19389 webserver.cc:460] Webserver started at http://127.18.71.65:43187/ using document root <none> and password file <none>
I20260809 07:48:19.378794 19389 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:19.378859 19389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:19.379132 19389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:19.381449 19389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "14bc8164b44243cc9bdb1028c2d45cbe"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "5e0e9393ff00e69ce6039c27656a47e4"
server_key_iv: "60b56929de9ba2faf5a4a3c0a47dfea6"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.382088 19389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "14bc8164b44243cc9bdb1028c2d45cbe"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "5e0e9393ff00e69ce6039c27656a47e4"
server_key_iv: "60b56929de9ba2faf5a4a3c0a47dfea6"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.386116 19389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:48:19.388656 19404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.389883 19389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:19.390079 19389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "14bc8164b44243cc9bdb1028c2d45cbe"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "5e0e9393ff00e69ce6039c27656a47e4"
server_key_iv: "60b56929de9ba2faf5a4a3c0a47dfea6"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.390182 19389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:19.414175 19389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:19.417832 19389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:19.418069 19389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:19.418720 19389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:19.419649 19389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:19.419701 19389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.419790 19389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:19.419824 19389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.429848 19389 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:46795
I20260809 07:48:19.429888 19525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:46795 every 8 connection(s)
I20260809 07:48:19.430832 19389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:19.433807 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 19389
I20260809 07:48:19.433909 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:19.434206 18717 external_mini_cluster.cc:1428] Setting key 7424b9b9d52accb6cc29b60d4f406dce
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261699, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:19.446110 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.432751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:43775 (local address 127.18.71.126:45283)
0809 07:48:19.432891 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.432895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.433836 (+ 941us) server_negotiation.cc:407] Connection header received
0809 07:48:19.434954 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.434958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.435018 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.435122 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:19.436656 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.437246 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.438651 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.439081 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.439222 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.442178 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:19.442182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:19.442185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:19.442209 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:19.443752 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.444342 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.444347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.444349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.444442 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.444880 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.444885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.444887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.445106 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:19.445235 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.445766 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.445906 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:48:19.447058 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.433111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45283 (local address 127.18.71.65:43775)
0809 07:48:19.433630 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:19.433690 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:19.434744 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:19.435318 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:19.435327 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:19.435767 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:19.436483 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.436508 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.437394 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.437401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.438502 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.438512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.439242 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.439244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.439381 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.439974 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:19.439990 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:19.441984 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:19.443887 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:19.443893 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:19.443909 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:19.444218 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:19.444606 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:19.444609 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:19.444612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:19.444769 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:19.445229 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:19.445236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:19.445509 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 07:48:19.445720 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 07:48:19.446065 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":126,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
I20260809 07:48:19.448510 19526 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45283
I20260809 07:48:19.448762 19526 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:19.449255 19526 heartbeater.cc:511] Master 127.18.71.126:45283 requested a full tablet report, sending...
I20260809 07:48:19.451017 19322 ts_manager.cc:194] Registered new tserver with Master: 14bc8164b44243cc9bdb1028c2d45cbe (127.18.71.65:46795)
I20260809 07:48:19.452231 19322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:43775
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:19.465623 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45283
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:19.581071 19531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:19.581328 19531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:19.581373 19531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:19.585398 19531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:19.585456 19531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:19.585515 19531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:19.590782 19531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:19.591833 19531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:19.592864 19531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:19.602552 19539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:19.602653 19531 server_base.cc:1034] running on GCE node
W20260809 07:48:19.602561 19537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:19.602571 19536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:19.603258 19531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:19.603914 19531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:19.605120 19531 hybrid_clock.cc:648] HybridClock initialized: now 1786261699605090 us; error 41 us; skew 500 ppm
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261699, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:19.607990 19531 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:19.608989 19531 webserver.cc:460] Webserver started at http://127.18.71.66:39045/ using document root <none> and password file <none>
I20260809 07:48:19.609634 19531 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:19.609732 19531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:19.609993 19531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:19.612179 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "18c3d37b7e3b4b7992db29fcc4125ee2"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "6b5c2b3d48150ee83b8371f3e9f78ed9"
server_key_iv: "f48c7246d975f1cb71ff6bafd3d0b751"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.612902 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "18c3d37b7e3b4b7992db29fcc4125ee2"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "6b5c2b3d48150ee83b8371f3e9f78ed9"
server_key_iv: "f48c7246d975f1cb71ff6bafd3d0b751"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.616685 19531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:19.619134 19546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.620304 19531 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:19.620445 19531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "18c3d37b7e3b4b7992db29fcc4125ee2"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "6b5c2b3d48150ee83b8371f3e9f78ed9"
server_key_iv: "f48c7246d975f1cb71ff6bafd3d0b751"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.620635 19531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:19.636243 19531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:19.639688 19531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:19.639894 19531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:19.640545 19531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:19.641371 19531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:19.641448 19531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.641549 19531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:19.641602 19531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.650992 19531 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:45767
I20260809 07:48:19.651029 19668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:45767 every 8 connection(s)
I20260809 07:48:19.651906 19531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:19.653097 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 19531
I20260809 07:48:19.653223 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:19.653491 18717 external_mini_cluster.cc:1428] Setting key 41760117623f24c211a95bd9c3dda4f3
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261699, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:48:19.666945 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.653786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:33561 (local address 127.18.71.126:45283)
0809 07:48:19.653942 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.653948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.655203 (+ 1255us) server_negotiation.cc:407] Connection header received
0809 07:48:19.656054 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.656060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.656119 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.656221 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:19.657786 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.658338 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.660102 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.660317 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.660441 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.663291 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:19.663295 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:19.663298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:19.663329 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:19.665044 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.665596 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.665602 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.665607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.665703 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.665980 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.665982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.665983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.666143 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:19.666214 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.666724 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.666823 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 07:48:19.667847 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.654249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45283 (local address 127.18.71.66:33561)
0809 07:48:19.655035 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:19.655097 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:19.655849 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:19.656379 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:19.656385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:19.656880 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:19.657614 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.657640 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.658443 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.658450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.659971 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.659989 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.660528 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.660536 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.660782 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.661405 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:19.661433 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:19.663110 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:19.665174 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:19.665179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:19.665189 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:19.665492 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:19.665789 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:19.665791 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:19.665792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:19.665919 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:19.666227 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:19.666232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:19.666518 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 07:48:19.666677 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 07:48:19.666915 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":114,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:48:19.669201 19670 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45283
I20260809 07:48:19.669456 19670 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:19.670053 19670 heartbeater.cc:511] Master 127.18.71.126:45283 requested a full tablet report, sending...
I20260809 07:48:19.671164 19322 ts_manager.cc:194] Registered new tserver with Master: 18c3d37b7e3b4b7992db29fcc4125ee2 (127.18.71.66:45767)
I20260809 07:48:19.671729 19322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:33561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:19.683928 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45283
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:19.797758 19675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:19.798066 19675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:19.798166 19675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:19.802235 19675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:19.802325 19675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:19.802413 19675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:19.807796 19675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45283
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:19.809031 19675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:19.810196 19675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:19.820429 19683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:19.820502 19675 server_base.cc:1034] running on GCE node
W20260809 07:48:19.820441 19681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:19.820422 19685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:19.821069 19675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:19.821686 19675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:19.822872 19675 hybrid_clock.cc:648] HybridClock initialized: now 1786261699822835 us; error 48 us; skew 500 ppm
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261699, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:19.825834 19675 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:19.826637 19675 webserver.cc:460] Webserver started at http://127.18.71.67:46445/ using document root <none> and password file <none>
I20260809 07:48:19.827189 19675 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:19.827250 19675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:19.827420 19675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:19.829463 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "54bc5e95797c44ce9de091472f4d3244"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "40c12cc4926fbeb02dc82b38801a2e1b"
server_key_iv: "8019681978fa85cd81a6d2dce5bf7aac"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.830045 19675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "54bc5e95797c44ce9de091472f4d3244"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "40c12cc4926fbeb02dc82b38801a2e1b"
server_key_iv: "8019681978fa85cd81a6d2dce5bf7aac"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.833679 19675 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:19.836036 19693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.837320 19675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:19.837416 19675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "54bc5e95797c44ce9de091472f4d3244"
format_stamp: "Formatted at 2026-08-09 07:48:19 on dist-test-slave-10pc"
server_key: "40c12cc4926fbeb02dc82b38801a2e1b"
server_key_iv: "8019681978fa85cd81a6d2dce5bf7aac"
server_key_version: "encryptionkey@0"
I20260809 07:48:19.837548 19675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:19.859310 19675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:19.862365 19675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:19.862571 19675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:19.863196 19675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:19.864038 19675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:19.864086 19675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.864153 19675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:19.864202 19675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:19.873016 19675 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:36895
I20260809 07:48:19.873095 19816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:36895 every 8 connection(s)
I20260809 07:48:19.873967 19675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:19.880753 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 19675
I20260809 07:48:19.880874 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:19.881158 18717 external_mini_cluster.cc:1428] Setting key 6aeb06eeb845949a07e20112aa300431
Aug 09 07:48:19 dist-test-slave-10pc krb5kdc[19274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261699, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:19.887207 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.875817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:40295 (local address 127.18.71.126:45283)
0809 07:48:19.875960 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.875964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.876791 (+ 827us) server_negotiation.cc:407] Connection header received
0809 07:48:19.877686 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.877690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.877747 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.877852 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:19.879009 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.879542 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.880705 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.881066 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.881173 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.883912 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:19.883915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:19.883918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:19.883942 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:19.885252 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.885744 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.885747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.885749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.885829 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.886165 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.886168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.886170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.886344 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:19.886451 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.886939 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.887054 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:48:19.887965 19819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.876150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45283 (local address 127.18.71.67:40295)
0809 07:48:19.876648 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:19.876690 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:19.877482 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:19.878047 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:19.878054 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:19.878397 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:19.878861 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.878874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.879673 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.879676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.880575 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.880584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.881224 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.881229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.881399 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.882019 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:19.882052 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:19.883756 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:19.885367 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:19.885373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:19.885387 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:19.885636 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:19.885920 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:19.885924 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:19.885925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:19.886066 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:19.886450 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:19.886457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:19.886699 (+ 242us) client_negotiation.cc:769] Sending connection context
0809 07:48:19.886897 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 07:48:19.887209 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":110,"threads_started":1}
I20260809 07:48:19.889104 19817 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45283
I20260809 07:48:19.889305 19817 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:19.889734 19817 heartbeater.cc:511] Master 127.18.71.126:45283 requested a full tablet report, sending...
I20260809 07:48:19.890671 19322 ts_manager.cc:194] Registered new tserver with Master: 54bc5e95797c44ce9de091472f4d3244 (127.18.71.67:36895)
I20260809 07:48:19.891216 19322 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:40295
I20260809 07:48:19.895013 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:48:19.905814 19365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.899016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58272 (local address 127.18.71.126:45283)
0809 07:48:19.899144 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.899149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.899305 (+ 156us) server_negotiation.cc:407] Connection header received
0809 07:48:19.899386 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.899388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.899423 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.899479 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:19.900093 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.900627 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.901403 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.901601 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.901683 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.902917 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:19.902920 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:19.902922 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:19.902941 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:19.904353 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.904763 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.904766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.904767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.904839 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:19.905077 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:19.905079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:19.905081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:19.905272 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:19.905370 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.905534 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.905618 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:48:19.911038 19322 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:19.914326 19322 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:19.930353 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.924521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:45767 (local address 127.0.0.1:51104)
0809 07:48:19.925036 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:19.925050 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:19.925143 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:19.925685 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:19.925691 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:19.925715 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:19.925857 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.925868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.927404 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.927407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.928956 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.928963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.929764 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.929768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.929890 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.929931 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 07:48:19.930049 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 07:48:19.930194 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":253,"thread_start_us":91,"threads_started":1}
I20260809 07:48:19.930353 19835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.924753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51104 (local address 127.18.71.66:45767)
0809 07:48:19.925267 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.925276 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.925296 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:48:19.925359 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.925362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.925534 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.925650 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:19.925995 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.927242 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.929080 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.929636 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.929762 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.929961 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.930078 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.930184 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":161,"threads_started":1}
I20260809 07:48:19.930394 19832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.924203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:36895 (local address 127.0.0.1:37880)
0809 07:48:19.924802 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:19.924815 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:19.924906 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:19.926327 (+ 1421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:19.926329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:19.926346 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:19.926440 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.926447 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.927581 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.927584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.928744 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.928754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.929869 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.929873 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.929979 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.930003 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:48:19.930053 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 07:48:19.930190 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":102,"thread_start_us":91,"threads_started":1}
I20260809 07:48:19.931152 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.923957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50128 (local address 127.18.71.65:46795)
0809 07:48:19.925339 (+ 1382us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.925353 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.925371 (+ 18us) server_negotiation.cc:407] Connection header received
0809 07:48:19.925431 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.925434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.925654 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.925824 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:19.926066 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.927241 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.929074 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.929855 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.929949 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.930107 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.930902 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.931014 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1296,"thread_start_us":108,"threads_started":1}
I20260809 07:48:19.931243 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.924805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.18.71.67:36895)
0809 07:48:19.925934 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:19.925944 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:19.925963 (+ 19us) server_negotiation.cc:407] Connection header received
0809 07:48:19.926050 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:19.926054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:19.926190 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:19.926345 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:19.926566 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.927436 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.928878 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.929717 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.930811 (+ 1094us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.930964 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:19.931008 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 07:48:19.931085 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":127,"threads_started":1}
I20260809 07:48:19.931532 19829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:19.923745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:46795 (local address 127.0.0.1:50128)
0809 07:48:19.924292 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:19.924344 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:19.924494 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:19.925824 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:19.925827 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:19.925845 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:19.925942 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.925948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.927404 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.927407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.928929 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:19.928940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:19.930171 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:19.930175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:19.930257 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:19.930280 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:48:19.930349 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 07:48:19.931434 (+ 1085us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":46,"thread_start_us":109,"threads_started":1}
I20260809 07:48:19.933663 19595 tablet_service.cc:1467] Processing CreateTablet for tablet a6483f3b41ef4275a6a17cb042a5ab9d (DEFAULT_TABLE table=test-table [id=fd7a430058e34826ab501fd1df75a4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:19.933826 19456 tablet_service.cc:1467] Processing CreateTablet for tablet a6483f3b41ef4275a6a17cb042a5ab9d (DEFAULT_TABLE table=test-table [id=fd7a430058e34826ab501fd1df75a4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:19.934966 19456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6483f3b41ef4275a6a17cb042a5ab9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:19.934966 19595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6483f3b41ef4275a6a17cb042a5ab9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:19.936918 19745 tablet_service.cc:1467] Processing CreateTablet for tablet a6483f3b41ef4275a6a17cb042a5ab9d (DEFAULT_TABLE table=test-table [id=fd7a430058e34826ab501fd1df75a4b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:19.938011 19745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6483f3b41ef4275a6a17cb042a5ab9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:19.941684 19837 tablet_bootstrap.cc:492] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2: Bootstrap starting.
I20260809 07:48:19.942958 19838 tablet_bootstrap.cc:492] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe: Bootstrap starting.
I20260809 07:48:19.945143 19838 tablet_bootstrap.cc:654] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:19.945480 19837 tablet_bootstrap.cc:654] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:19.945632 19839 tablet_bootstrap.cc:492] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244: Bootstrap starting.
I20260809 07:48:19.946345 19838 log.cc:824] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:19.946359 19837 log.cc:824] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:19.947561 19839 tablet_bootstrap.cc:654] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:19.948258 19838 tablet_bootstrap.cc:492] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe: No bootstrap required, opened a new log
I20260809 07:48:19.948510 19838 ts_tablet_manager.cc:1397] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:48:19.948705 19839 log.cc:824] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:19.948803 19837 tablet_bootstrap.cc:492] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2: No bootstrap required, opened a new log
I20260809 07:48:19.948972 19837 ts_tablet_manager.cc:1397] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260809 07:48:19.950605 19839 tablet_bootstrap.cc:492] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244: No bootstrap required, opened a new log
I20260809 07:48:19.950781 19839 ts_tablet_manager.cc:1397] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 07:48:19.951676 19838 raft_consensus.cc:348] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:19.951946 19838 raft_consensus.cc:374] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:19.952003 19838 raft_consensus.cc:729] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14bc8164b44243cc9bdb1028c2d45cbe, State: Initialized, Role: FOLLOWER
I20260809 07:48:19.952541 19838 consensus_queue.cc:260] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:19.952690 19837 raft_consensus.cc:348] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:19.952953 19837 raft_consensus.cc:374] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:19.953008 19837 raft_consensus.cc:729] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c3d37b7e3b4b7992db29fcc4125ee2, State: Initialized, Role: FOLLOWER
I20260809 07:48:19.953362 19526 heartbeater.cc:503] Master 127.18.71.126:45283 was elected leader, sending a full tablet report...
I20260809 07:48:19.953449 19837 consensus_queue.cc:260] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:19.954128 19670 heartbeater.cc:503] Master 127.18.71.126:45283 was elected leader, sending a full tablet report...
I20260809 07:48:19.954409 19837 ts_tablet_manager.cc:1428] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 07:48:19.954818 19838 ts_tablet_manager.cc:1428] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:48:19.955271 19839 raft_consensus.cc:348] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:19.955509 19839 raft_consensus.cc:374] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:19.955560 19839 raft_consensus.cc:729] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54bc5e95797c44ce9de091472f4d3244, State: Initialized, Role: FOLLOWER
I20260809 07:48:19.955962 19839 consensus_queue.cc:260] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:19.956715 19817 heartbeater.cc:503] Master 127.18.71.126:45283 was elected leader, sending a full tablet report...
I20260809 07:48:19.957077 19839 ts_tablet_manager.cc:1428] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:48:20.064648 19844 raft_consensus.cc:482] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:20.064877 19844 raft_consensus.cc:504] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:20.066016 19844 leader_election.cc:288] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14bc8164b44243cc9bdb1028c2d45cbe (127.18.71.65:46795), 54bc5e95797c44ce9de091472f4d3244 (127.18.71.67:36895)
I20260809 07:48:20.070112 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:20.066417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:45015 (local address 127.18.71.65:46795)
0809 07:48:20.066557 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:20.066562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:20.066903 (+ 341us) server_negotiation.cc:407] Connection header received
0809 07:48:20.067009 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:20.067011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:20.067059 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:20.067139 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:20.067434 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.067965 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.069119 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.069533 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.069626 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:20.069709 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:20.069905 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 07:48:20.069983 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:48:20.070112 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:20.066317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:36895 (local address 127.18.71.66:34361)
0809 07:48:20.066532 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:20.066548 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:20.066676 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:20.066995 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:20.066998 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:20.067017 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:20.067167 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:20.067175 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.067992 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.067998 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:20.069001 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:20.069011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.069734 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.069738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:20.069824 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:20.069841 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 07:48:20.069940 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 07:48:20.070003 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":88}
I20260809 07:48:20.070115 19846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:20.066317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:46795 (local address 127.18.71.66:45015)
0809 07:48:20.066784 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:20.066799 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:20.066917 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:20.067159 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:20.067162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:20.067176 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:20.067310 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:20.067317 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.068081 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.068083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:20.068988 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:20.068997 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.069661 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.069668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:20.069759 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:20.069791 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 07:48:20.069863 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 07:48:20.069939 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":35,"thread_start_us":142,"threads_started":1}
I20260809 07:48:20.070191 19836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:20.066423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:34361 (local address 127.18.71.67:36895)
0809 07:48:20.066549 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:20.066555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:20.066611 (+ 56us) server_negotiation.cc:407] Connection header received
0809 07:48:20.066803 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:20.066809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:20.066875 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:20.066959 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:20.067302 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.067869 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.069161 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.069599 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.069713 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:20.069765 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:20.069980 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 07:48:20.070051 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:48:20.070876 19767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6483f3b41ef4275a6a17cb042a5ab9d" candidate_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54bc5e95797c44ce9de091472f4d3244" is_pre_election: true
I20260809 07:48:20.070874 19476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6483f3b41ef4275a6a17cb042a5ab9d" candidate_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" is_pre_election: true
I20260809 07:48:20.071336 19476 raft_consensus.cc:2454] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c3d37b7e3b4b7992db29fcc4125ee2 in term 0.
I20260809 07:48:20.071921 19547 leader_election.cc:302] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14bc8164b44243cc9bdb1028c2d45cbe, 18c3d37b7e3b4b7992db29fcc4125ee2; no voters:
I20260809 07:48:20.072168 19844 raft_consensus.cc:2781] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:20.072294 19844 raft_consensus.cc:482] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:20.072347 19844 raft_consensus.cc:3037] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:20.073145 19767 raft_consensus.cc:2454] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18c3d37b7e3b4b7992db29fcc4125ee2 in term 0.
I20260809 07:48:20.073457 19844 raft_consensus.cc:504] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:20.073824 19844 leader_election.cc:288] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [CANDIDATE]: Term 1 election: Requested vote from peers 14bc8164b44243cc9bdb1028c2d45cbe (127.18.71.65:46795), 54bc5e95797c44ce9de091472f4d3244 (127.18.71.67:36895)
I20260809 07:48:20.074210 19767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6483f3b41ef4275a6a17cb042a5ab9d" candidate_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54bc5e95797c44ce9de091472f4d3244"
I20260809 07:48:20.074219 19476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6483f3b41ef4275a6a17cb042a5ab9d" candidate_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14bc8164b44243cc9bdb1028c2d45cbe"
I20260809 07:48:20.074340 19767 raft_consensus.cc:3037] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:20.074345 19476 raft_consensus.cc:3037] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:20.075629 19476 raft_consensus.cc:2454] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c3d37b7e3b4b7992db29fcc4125ee2 in term 1.
I20260809 07:48:20.075629 19767 raft_consensus.cc:2454] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18c3d37b7e3b4b7992db29fcc4125ee2 in term 1.
I20260809 07:48:20.075919 19547 leader_election.cc:302] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 14bc8164b44243cc9bdb1028c2d45cbe, 18c3d37b7e3b4b7992db29fcc4125ee2, 54bc5e95797c44ce9de091472f4d3244; no voters:
I20260809 07:48:20.076133 19844 raft_consensus.cc:2781] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:20.076349 19844 raft_consensus.cc:686] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [term 1 LEADER]: Becoming Leader. State: Replica: 18c3d37b7e3b4b7992db29fcc4125ee2, State: Running, Role: LEADER
I20260809 07:48:20.076695 19844 consensus_queue.cc:237] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } }
I20260809 07:48:20.079957 19319 catalog_manager.cc:5495] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18c3d37b7e3b4b7992db29fcc4125ee2 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14bc8164b44243cc9bdb1028c2d45cbe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18c3d37b7e3b4b7992db29fcc4125ee2" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54bc5e95797c44ce9de091472f4d3244" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36895 } health_report { overall_health: UNKNOWN } } }
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:20.124620 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:20.111089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50130 (local address 127.18.71.65:46795)
0809 07:48:20.111270 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:20.111276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:20.111442 (+ 166us) server_negotiation.cc:407] Connection header received
0809 07:48:20.111608 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:20.111611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:20.111667 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:20.111756 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:20.112475 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.113100 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.113948 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.114177 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.114289 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:20.121139 (+ 6850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:20.121154 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:20.121170 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:20.121196 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:20.122720 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:20.123176 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:20.123182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:20.123183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:20.123264 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:20.123540 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:20.123545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:20.123549 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:20.123817 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:20.124099 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:20.124270 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 07:48:20.124400 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 07:48:20.125165 19818 tablet.cc:2369] T a6483f3b41ef4275a6a17cb042a5ab9d P 54bc5e95797c44ce9de091472f4d3244: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261700, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
W20260809 07:48:20.153064 19671 tablet.cc:2369] T a6483f3b41ef4275a6a17cb042a5ab9d P 18c3d37b7e3b4b7992db29fcc4125ee2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:20.156342 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:20.147721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.18.71.65:46795)
0809 07:48:20.147892 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:20.147897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:20.147954 (+ 57us) server_negotiation.cc:407] Connection header received
0809 07:48:20.148071 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:20.148073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:20.148116 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:20.148193 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:20.148940 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.149472 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.150287 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.150553 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.150654 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:20.153344 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:20.153348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:20.153351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:20.153380 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:20.154710 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:20.155152 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:20.155154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:20.155155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:20.155228 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:20.155485 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:20.155487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:20.155488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:20.155642 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:20.155734 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:20.156123 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 07:48:20.156228 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
W20260809 07:48:20.182221 19527 tablet.cc:2369] T a6483f3b41ef4275a6a17cb042a5ab9d P 14bc8164b44243cc9bdb1028c2d45cbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:20.183382 19830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:20.175035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.18.71.65:46795)
0809 07:48:20.175194 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:20.175199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:20.175347 (+ 148us) server_negotiation.cc:407] Connection header received
0809 07:48:20.175534 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:20.175537 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:20.175581 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:20.175667 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:20.176358 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.176926 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.177832 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:20.178061 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:20.178171 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:20.180535 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:20.180538 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:20.180540 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:20.180563 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:20.181722 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:20.182153 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:20.182157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:20.182159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:20.182237 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:20.182549 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:20.182552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:20.182554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:20.182716 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:20.182850 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:20.183109 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 07:48:20.183225 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 07:48:20.185158 19433 tablet_service.cc:2908] Rejecting scan request for tablet a6483f3b41ef4275a6a17cb042a5ab9d: Uninitialized: clean time has not yet been initialized
I20260809 07:48:20.187568 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 19389
I20260809 07:48:20.196406 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 19531
I20260809 07:48:20.203439 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 19675
I20260809 07:48:20.210389 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 19287
2026-08-09T07:48:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2449 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19864](info): set up 2 sockets
Aug 09 07:48:20 dist-test-slave-10pc krb5kdc[19864](info): commencing operation
krb5kdc: starting...
W20260809 07:48:21.292661 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:21 dist-test-slave-10pc krb5kdc[19864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:21.422829 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40885
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:40885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:21.534152 19878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:21.534461 19878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:21.534544 19878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:21.538585 19878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:21.538674 19878 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 07:48:21.538710 19878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:21.538770 19878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:21.538810 19878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:21.538846 19878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:21.544085 19878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:40885
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:21.545226 19878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:21.546252 19878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:48:21.554710 19878 server_base.cc:1034] running on GCE node
W20260809 07:48:21.554657 19887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:21.554731 19884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:21.554664 19885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:21.555536 19878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:21.556737 19878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:21.557898 19878 hybrid_clock.cc:648] HybridClock initialized: now 1786261701557868 us; error 43 us; skew 500 ppm
Aug 09 07:48:21 dist-test-slave-10pc krb5kdc[19864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261701, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:21.560732 19878 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:21.561551 19878 webserver.cc:460] Webserver started at http://127.18.71.126:33987/ using document root <none> and password file <none>
I20260809 07:48:21.562076 19878 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:21.562136 19878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:21.562301 19878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:21.564272 19878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "27178697edc9420597ecd386988dc5b7"
format_stamp: "Formatted at 2026-08-09 07:48:21 on dist-test-slave-10pc"
server_key: "bb7ad5e99795018cd93f3cf5e013cfe5"
server_key_iv: "4c3b5357cb7f949cfd3f39ce44a07119"
server_key_version: "encryptionkey@0"
I20260809 07:48:21.564883 19878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "27178697edc9420597ecd386988dc5b7"
format_stamp: "Formatted at 2026-08-09 07:48:21 on dist-test-slave-10pc"
server_key: "bb7ad5e99795018cd93f3cf5e013cfe5"
server_key_iv: "4c3b5357cb7f949cfd3f39ce44a07119"
server_key_version: "encryptionkey@0"
I20260809 07:48:21.568387 19878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:21.570736 19895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:21.571936 19878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:21.572064 19878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "27178697edc9420597ecd386988dc5b7"
format_stamp: "Formatted at 2026-08-09 07:48:21 on dist-test-slave-10pc"
server_key: "bb7ad5e99795018cd93f3cf5e013cfe5"
server_key_iv: "4c3b5357cb7f949cfd3f39ce44a07119"
server_key_version: "encryptionkey@0"
I20260809 07:48:21.572171 19878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:21.591041 19878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:21.594672 19878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:21.594879 19878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:21.602975 19878 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:40885
I20260809 07:48:21.603052 19961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:40885 every 8 connection(s)
I20260809 07:48:21.603971 19878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:21.606866 19962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:21.609109 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 19878
I20260809 07:48:21.609228 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:21.609506 18717 external_mini_cluster.cc:1428] Setting key 9150ffc3bdbf2ba6f31516dfca39e5cf
I20260809 07:48:21.614331 19962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: Bootstrap starting.
I20260809 07:48:21.617041 19962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:21.617784 19962 log.cc:824] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:48:21 dist-test-slave-10pc krb5kdc[19864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:21.619839 19962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: No bootstrap required, opened a new log
I20260809 07:48:21.624365 19962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27178697edc9420597ecd386988dc5b7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40885 } }
I20260809 07:48:21.624696 19962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:21.624760 19962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27178697edc9420597ecd386988dc5b7, State: Initialized, Role: FOLLOWER
I20260809 07:48:21.625267 19962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27178697edc9420597ecd386988dc5b7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40885 } }
I20260809 07:48:21.625425 19962 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:21.625490 19962 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:21.625591 19962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:21.626232 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:21.610892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.18.71.126:40885)
0809 07:48:21.611310 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:21.611330 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:21.611375 (+ 45us) server_negotiation.cc:407] Connection header received
0809 07:48:21.612376 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:21.612401 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:21.612856 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:21.613222 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:21.613897 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.615354 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.616209 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.616917 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.617092 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:21.619618 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:21.619624 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:21.619641 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:21.619670 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:21.622902 (+ 3232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:21.623324 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:21.623330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:21.623336 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:21.623478 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:21.623740 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:21.623745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:21.623747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:21.624295 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:21.624518 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:21.624829 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 07:48:21.625205 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":117,"threads_started":1}
I20260809 07:48:21.626700 19962 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27178697edc9420597ecd386988dc5b7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40885 } }
I20260809 07:48:21.627046 19962 leader_election.cc:302] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27178697edc9420597ecd386988dc5b7; no voters:
I20260809 07:48:21.627372 19962 leader_election.cc:288] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:21.627467 19967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:21.627693 19967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [term 1 LEADER]: Becoming Leader. State: Replica: 27178697edc9420597ecd386988dc5b7, State: Running, Role: LEADER
I20260809 07:48:21.628104 19967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27178697edc9420597ecd386988dc5b7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40885 } }
I20260809 07:48:21.628734 19962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:21.630188 19968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27178697edc9420597ecd386988dc5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27178697edc9420597ecd386988dc5b7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40885 } } }
I20260809 07:48:21.630312 19968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:21.630640 19969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27178697edc9420597ecd386988dc5b7. Latest consensus state: current_term: 1 leader_uuid: "27178697edc9420597ecd386988dc5b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27178697edc9420597ecd386988dc5b7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40885 } } }
I20260809 07:48:21.631351 19969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7 [sys.catalog]: This master's current role is: LEADER
W20260809 07:48:21.633872 19984 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:21.633950 19984 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:21.634025 19979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:21.634874 19979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:21.640894 19979 catalog_manager.cc:1347] Generated new cluster ID: 532d8493e2b3418bb404e7263ff37ea2
I20260809 07:48:21.640962 19979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:21.650152 19979 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:21.651059 19979 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:21.657891 19979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: Generated new TSK 0
I20260809 07:48:21.658710 19979 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:21.692289 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:40885
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:21.806187 19990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:21.806424 19990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:21.806492 19990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:21.810467 19990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:21.810520 19990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:21.810580 19990 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:21.810647 19990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:21.815802 19990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:21.816907 19990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:21.817931 19990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:21.827634 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:21.827643 19995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:21.827643 19997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:21.827819 19990 server_base.cc:1034] running on GCE node
I20260809 07:48:21.828305 19990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:21.829102 19990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:21.830273 19990 hybrid_clock.cc:648] HybridClock initialized: now 1786261701830242 us; error 42 us; skew 500 ppm
Aug 09 07:48:21 dist-test-slave-10pc krb5kdc[19864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261701, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:21.833170 19990 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:21.833998 19990 webserver.cc:460] Webserver started at http://127.18.71.65:43439/ using document root <none> and password file <none>
I20260809 07:48:21.834618 19990 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:21.834681 19990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:21.834923 19990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:21.837025 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "9fd180c4b7154c15a2e541ddbd6418aa"
format_stamp: "Formatted at 2026-08-09 07:48:21 on dist-test-slave-10pc"
server_key: "cf45a5bf97ebcd00e50cdbba8f2047b9"
server_key_iv: "b12343a09a39c3992e1a680edb6ec4c3"
server_key_version: "encryptionkey@0"
I20260809 07:48:21.837590 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "9fd180c4b7154c15a2e541ddbd6418aa"
format_stamp: "Formatted at 2026-08-09 07:48:21 on dist-test-slave-10pc"
server_key: "cf45a5bf97ebcd00e50cdbba8f2047b9"
server_key_iv: "b12343a09a39c3992e1a680edb6ec4c3"
server_key_version: "encryptionkey@0"
I20260809 07:48:21.841286 19990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:48:21.843650 20006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:21.844887 19990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 07:48:21.845017 19990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "9fd180c4b7154c15a2e541ddbd6418aa"
format_stamp: "Formatted at 2026-08-09 07:48:21 on dist-test-slave-10pc"
server_key: "cf45a5bf97ebcd00e50cdbba8f2047b9"
server_key_iv: "b12343a09a39c3992e1a680edb6ec4c3"
server_key_version: "encryptionkey@0"
I20260809 07:48:21.845119 19990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:21.866285 19990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:21.869614 19990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:21.869763 19990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:21.871191 19990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:21.871239 19990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:21.871305 19990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:21.871336 19990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:21 dist-test-slave-10pc krb5kdc[19864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261701, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:21.882407 19990 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:42801
I20260809 07:48:21.882462 20128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:42801 every 8 connection(s)
I20260809 07:48:21.883306 19990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:21.885386 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:21.872105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:38949 (local address 127.18.71.126:40885)
0809 07:48:21.872240 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:21.872244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:21.873358 (+ 1114us) server_negotiation.cc:407] Connection header received
0809 07:48:21.874554 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:21.874558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:21.874627 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:21.874733 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:21.876383 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.877312 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.878610 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.878885 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.879009 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:21.881587 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:21.881591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:21.881594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:21.881622 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:21.883177 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:21.883749 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:21.883755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:21.883756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:21.883872 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:21.884272 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:21.884276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:21.884277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:21.884509 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:21.884603 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:21.885090 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 07:48:21.885216 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:48:21.886085 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:21.872704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40885 (local address 127.18.71.65:38949)
0809 07:48:21.873193 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:21.873234 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:21.874247 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:21.874956 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:21.874963 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:21.875459 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:21.876133 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:21.876154 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.877437 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.877440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:21.878473 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:21.878483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.879053 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.879057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:21.879230 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:21.879796 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:21.879812 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:21.881423 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:21.883293 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:21.883298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:21.883311 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:21.883633 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:21.883986 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:21.883988 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:21.883990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:21.884153 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:21.884609 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:21.884618 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:21.884839 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 07:48:21.885038 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 07:48:21.885335 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":143,"threads_started":1}
I20260809 07:48:21.887372 20129 heartbeater.cc:347] Connected to a master server at 127.18.71.126:40885
I20260809 07:48:21.887526 20129 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:21.888051 20129 heartbeater.cc:511] Master 127.18.71.126:40885 requested a full tablet report, sending...
I20260809 07:48:21.889011 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 19990
I20260809 07:48:21.889145 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:21.889472 18717 external_mini_cluster.cc:1428] Setting key e56f8f95bdc1e72acf26f190a50a6d93
I20260809 07:48:21.889781 19912 ts_manager.cc:194] Registered new tserver with Master: 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801)
I20260809 07:48:21.891060 19912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:38949
I20260809 07:48:21.897459 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:21.889659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:51039 (local address 127.18.71.126:40885)
0809 07:48:21.889771 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:21.889775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:21.890561 (+ 786us) server_negotiation.cc:407] Connection header received
0809 07:48:21.890608 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:21.890610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:21.890652 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:21.890718 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:21.891129 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.891689 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.892850 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.893159 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.893231 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:21.894372 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:21.894375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:21.894377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:21.894399 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:21.895839 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:21.896263 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:21.896266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:21.896268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:21.896346 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:21.896649 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:21.896653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:21.896654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:21.896817 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:21.896922 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:21.897201 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 07:48:21.897307 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 07:48:21.897480 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:21.889576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40885 (local address 127.18.71.65:51039)
0809 07:48:21.889974 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:21.889985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:21.890090 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:21.890738 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:21.890740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:21.890922 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:21.891011 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:21.891016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.891799 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.891802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:21.892719 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:21.892729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:21.893373 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:21.893376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:21.893467 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:21.893932 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:21.893942 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:21.894242 (+ 300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:21.895956 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:21.895960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:21.895962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:21.896161 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:21.896438 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:21.896441 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:21.896443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:21.896552 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:21.896913 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:21.896916 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:21.897010 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 07:48:21.897143 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 07:48:21.897283 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":16,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:21.919260 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:40885
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:22.036113 20138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:22.036371 20138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:22.036453 20138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:22.042402 20138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:22.042484 20138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:22.042537 20138 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:22.042583 20138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:22.050164 20138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:22.051499 20138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:22.052688 20138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:22.062626 20144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:22.062649 20143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:22.062704 20146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:22.063031 20138 server_base.cc:1034] running on GCE node
I20260809 07:48:22.063426 20138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:22.064105 20138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:22.065300 20138 hybrid_clock.cc:648] HybridClock initialized: now 1786261702065254 us; error 65 us; skew 500 ppm
Aug 09 07:48:22 dist-test-slave-10pc krb5kdc[19864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261702, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:22.068143 20138 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:22.069250 20138 webserver.cc:460] Webserver started at http://127.18.71.66:33361/ using document root <none> and password file <none>
I20260809 07:48:22.069821 20138 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:22.069882 20138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:22.070104 20138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:22.072281 20138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "4454b16ecd7e4edd8354928b7375e7f5"
format_stamp: "Formatted at 2026-08-09 07:48:22 on dist-test-slave-10pc"
server_key: "680ec4387f85b72bad114f66afe6e24b"
server_key_iv: "9cc6e00a3ff1d909a5bd635a36f36bb2"
server_key_version: "encryptionkey@0"
I20260809 07:48:22.072935 20138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "4454b16ecd7e4edd8354928b7375e7f5"
format_stamp: "Formatted at 2026-08-09 07:48:22 on dist-test-slave-10pc"
server_key: "680ec4387f85b72bad114f66afe6e24b"
server_key_iv: "9cc6e00a3ff1d909a5bd635a36f36bb2"
server_key_version: "encryptionkey@0"
I20260809 07:48:22.077131 20138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:48:22.079712 20154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:22.081076 20138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:22.081223 20138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "4454b16ecd7e4edd8354928b7375e7f5"
format_stamp: "Formatted at 2026-08-09 07:48:22 on dist-test-slave-10pc"
server_key: "680ec4387f85b72bad114f66afe6e24b"
server_key_iv: "9cc6e00a3ff1d909a5bd635a36f36bb2"
server_key_version: "encryptionkey@0"
I20260809 07:48:22.081332 20138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:22.098109 20138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:22.101374 20138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:22.101575 20138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:22.103071 20138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:22.103147 20138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:22.103214 20138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:22.103252 20138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:22 dist-test-slave-10pc krb5kdc[19864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261702, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:22.114925 20138 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:40475
I20260809 07:48:22.114977 20278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:40475 every 8 connection(s)
I20260809 07:48:22.115998 20138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:22.119308 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 20138
I20260809 07:48:22.119432 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:22.119606 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.104045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:38901 (local address 127.18.71.126:40885)
0809 07:48:22.104209 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.104215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.105317 (+ 1102us) server_negotiation.cc:407] Connection header received
0809 07:48:22.106782 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.106786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.106853 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.106919 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:22.108854 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.109779 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.111633 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.111986 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.112138 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.115468 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:22.115474 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:22.115479 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:22.115516 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:22.117492 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.117983 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.117987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.117990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.118089 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.118410 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.118414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.118417 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.118681 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:22.118785 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.119301 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.119427 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 07:48:22.119746 18717 external_mini_cluster.cc:1428] Setting key 4224ee1255af9d01873b654c85ccc861
I20260809 07:48:22.120404 20163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.104547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40885 (local address 127.18.71.66:38901)
0809 07:48:22.105148 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.105208 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.106503 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.107375 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.107382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.107891 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:22.108684 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.108711 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.109956 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.109961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.111488 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.111502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.112183 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.112189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.112421 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.113141 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:22.113163 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:22.115276 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:22.117613 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.117617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.117627 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.117873 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.118180 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.118182 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.118184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.118310 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.118798 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:22.118803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:22.119016 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.119223 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.119535 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":149,"threads_started":1}
I20260809 07:48:22.121979 20279 heartbeater.cc:347] Connected to a master server at 127.18.71.126:40885
I20260809 07:48:22.122303 20279 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:22.122948 20279 heartbeater.cc:511] Master 127.18.71.126:40885 requested a full tablet report, sending...
I20260809 07:48:22.124120 19911 ts_manager.cc:194] Registered new tserver with Master: 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66:40475)
I20260809 07:48:22.124966 19911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:38901
I20260809 07:48:22.132601 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.124507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:42543 (local address 127.18.71.126:40885)
0809 07:48:22.124622 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.124625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.124840 (+ 215us) server_negotiation.cc:407] Connection header received
0809 07:48:22.125020 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.125022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.125059 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.125144 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:22.125848 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.126345 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.127157 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.127371 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.127480 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.128710 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:22.128714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:22.128719 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:22.128745 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:22.130432 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.131003 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.131009 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.131012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.131112 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.131461 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.131466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.131468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.131765 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:22.131846 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.132230 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.132373 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:48:22.133360 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.124430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40885 (local address 127.18.71.66:42543)
0809 07:48:22.124758 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.124769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.124877 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.125276 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.125281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.125533 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:22.125698 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.125707 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.126473 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.126477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.127036 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.127044 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.127485 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.127489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.127598 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.128139 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:22.128158 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:22.128559 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:22.130595 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.130598 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.130604 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.130885 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.131241 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.131246 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.131250 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.131348 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.131929 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:22.131933 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:22.132037 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.132958 (+ 921us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.133132 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":143,"threads_started":1}
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:22.159614 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:40885
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:38745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:22.277639 20289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:22.277890 20289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:22.277935 20289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:22.281873 20289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:22.281927 20289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:22.281967 20289 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:22.281999 20289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:22.287277 20289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:38745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:40885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:22.288311 20289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:22.289335 20289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:22.298774 20297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:22.298774 20295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:22.298784 20294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:22.299069 20289 server_base.cc:1034] running on GCE node
I20260809 07:48:22.299558 20289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:22.300205 20289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:22.301400 20289 hybrid_clock.cc:648] HybridClock initialized: now 1786261702301369 us; error 39 us; skew 500 ppm
Aug 09 07:48:22 dist-test-slave-10pc krb5kdc[19864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261702, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:22.304116 20289 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:22.305042 20289 webserver.cc:460] Webserver started at http://127.18.71.67:37473/ using document root <none> and password file <none>
I20260809 07:48:22.305562 20289 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:22.305650 20289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:22.305869 20289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:22.307862 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "96e0ebfa3635415b984e927f487e2317"
format_stamp: "Formatted at 2026-08-09 07:48:22 on dist-test-slave-10pc"
server_key: "f5cc98eda185c7880b2d1c40182fdd1d"
server_key_iv: "ca79fde4838fdb34f785a885903b30c0"
server_key_version: "encryptionkey@0"
I20260809 07:48:22.308390 20289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "96e0ebfa3635415b984e927f487e2317"
format_stamp: "Formatted at 2026-08-09 07:48:22 on dist-test-slave-10pc"
server_key: "f5cc98eda185c7880b2d1c40182fdd1d"
server_key_iv: "ca79fde4838fdb34f785a885903b30c0"
server_key_version: "encryptionkey@0"
I20260809 07:48:22.311978 20289 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:22.314388 20305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:22.315567 20289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:22.315670 20289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "96e0ebfa3635415b984e927f487e2317"
format_stamp: "Formatted at 2026-08-09 07:48:22 on dist-test-slave-10pc"
server_key: "f5cc98eda185c7880b2d1c40182fdd1d"
server_key_iv: "ca79fde4838fdb34f785a885903b30c0"
server_key_version: "encryptionkey@0"
I20260809 07:48:22.315752 20289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:22.347826 20289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:22.351496 20289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:22.351670 20289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:22.353516 20289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:22.353587 20289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:22.353641 20289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:22.353670 20289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:22 dist-test-slave-10pc krb5kdc[19864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261702, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:22.364988 20289 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:45257
I20260809 07:48:22.365129 20426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:45257 every 8 connection(s)
I20260809 07:48:22.366039 20289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:22.366851 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 20289
I20260809 07:48:22.366979 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:22.367302 18717 external_mini_cluster.cc:1428] Setting key dfe6b2c78bafeda22107366a3205f737
I20260809 07:48:22.367914 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.354290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:40877 (local address 127.18.71.126:40885)
0809 07:48:22.354463 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.354468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.355761 (+ 1293us) server_negotiation.cc:407] Connection header received
0809 07:48:22.357016 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.357019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.357078 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.357190 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:22.358556 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.359081 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.360245 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.360587 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.360708 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.363794 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:22.363801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:22.363804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:22.363833 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:22.365565 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.366146 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.366149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.366151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.366251 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.366676 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.366682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.366685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.366925 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:22.367026 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.367594 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.367741 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 07:48:22.368947 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.355116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40885 (local address 127.18.71.67:40877)
0809 07:48:22.355587 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.355638 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.356802 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.357379 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.357386 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.357919 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:22.358401 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.358415 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.359216 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.359219 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.360104 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.360113 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.360761 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.360765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.360990 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.361619 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:22.361640 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:22.363589 (+ 1949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:22.365676 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.365682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.365696 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.366034 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.366350 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.366357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.366362 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.366554 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.367024 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:22.367033 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:22.367339 (+ 306us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.367557 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.367881 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":134,"threads_started":1}
I20260809 07:48:22.370329 20427 heartbeater.cc:347] Connected to a master server at 127.18.71.126:40885
I20260809 07:48:22.370483 20427 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:22.370966 20427 heartbeater.cc:511] Master 127.18.71.126:40885 requested a full tablet report, sending...
I20260809 07:48:22.372008 19912 ts_manager.cc:194] Registered new tserver with Master: 96e0ebfa3635415b984e927f487e2317 (127.18.71.67:45257)
I20260809 07:48:22.372928 19912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:40877
I20260809 07:48:22.379823 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.372814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:56241 (local address 127.18.71.126:40885)
0809 07:48:22.372927 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.372930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.373182 (+ 252us) server_negotiation.cc:407] Connection header received
0809 07:48:22.373309 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.373312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.373350 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.373442 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:22.373876 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.374376 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.375279 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.375532 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.375634 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.376773 (+ 1139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:22.376776 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:22.376779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:22.376802 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:22.378231 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.378653 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.378657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.378659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.378738 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.379014 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.379018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.379020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.379185 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:22.379293 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.379555 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.379660 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 07:48:22.379824 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.372745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40885 (local address 127.18.71.67:56241)
0809 07:48:22.373080 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.373096 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.373198 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.373459 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.373462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.373668 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:22.373761 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.373767 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.374536 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.374538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.375151 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.375159 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.375649 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.375653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.375759 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.376266 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:22.376278 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:22.376640 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:22.378345 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.378350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.378352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.378551 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.378829 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.378833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.378835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.378921 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.379284 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:22.379287 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:22.379385 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.379498 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.379641 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":99,"threads_started":1}
I20260809 07:48:22.381465 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 07:48:22 dist-test-slave-10pc krb5kdc[19864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261702, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 07:48:22 dist-test-slave-10pc krb5kdc[19864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261702, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:22.409009 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.400903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52792 (local address 127.18.71.126:40885)
0809 07:48:22.401034 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.401038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.401050 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:48:22.401092 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.401094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.401134 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.401232 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:22.401932 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.402460 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.403311 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.403527 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.403642 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.406082 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:22.406086 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:22.406089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:22.406113 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:22.407538 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.407892 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.407895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.407896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.407968 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.408190 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.408192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.408193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.408352 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:22.408446 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.408696 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.408789 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 07:48:22.411895 19912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:22.414947 19912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:22.429857 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.424026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.18.71.67:45257)
0809 07:48:22.424926 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.424938 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.424971 (+ 33us) server_negotiation.cc:407] Connection header received
0809 07:48:22.425048 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.425052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.425196 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.425345 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:22.425575 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.426691 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.428268 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.429107 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.429268 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.429491 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.429595 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.429698 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":115,"threads_started":1}
I20260809 07:48:22.429848 20447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.423407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:45257 (local address 127.0.0.1:60210)
0809 07:48:22.424278 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.424328 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.424495 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.425324 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.425327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.425345 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:22.425448 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.425455 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.426818 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.426821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.428096 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.428105 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.429264 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.429268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.429410 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.429458 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.429565 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.429683 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":87,"spinlock_wait_cycles":8192,"thread_start_us":101,"threads_started":1}
I20260809 07:48:22.430306 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.423928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:42801 (local address 127.0.0.1:56840)
0809 07:48:22.425558 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.425570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.425687 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.426066 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.426070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.426087 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:22.426200 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.426206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.427416 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.427420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.428617 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.428625 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.429790 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.429793 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.429896 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.429929 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.430104 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.430185 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1563,"mutex_wait_us":32,"thread_start_us":92,"threads_started":1}
I20260809 07:48:22.430653 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.423222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40475 (local address 127.0.0.1:47126)
0809 07:48:22.424278 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.424327 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.424496 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.425109 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.425115 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.425136 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:22.425275 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.425281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.426785 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.426788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.428353 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.428363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.430073 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.430077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.430164 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.430187 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.430437 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.430520 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":95,"spinlock_wait_cycles":3200,"thread_start_us":70,"threads_started":1}
I20260809 07:48:22.430742 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.424092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56840 (local address 127.18.71.65:42801)
0809 07:48:22.424649 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.424656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.425669 (+ 1013us) server_negotiation.cc:407] Connection header received
0809 07:48:22.425793 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.425798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.425944 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.426092 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:22.426317 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.427277 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.428802 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.429482 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.429599 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.429728 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.430504 (+ 776us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.430613 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":87,"threads_started":1}
I20260809 07:48:22.431149 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.423358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.18.71.66:40475)
0809 07:48:22.423717 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.423730 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.424409 (+ 679us) server_negotiation.cc:407] Connection header received
0809 07:48:22.424705 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.424711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.424930 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.425117 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:22.425399 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.426645 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.428496 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.429485 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.429594 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.429740 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.430867 (+ 1127us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.430969 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":147,"threads_started":1}
I20260809 07:48:22.433027 20361 tablet_service.cc:1467] Processing CreateTablet for tablet 26318310266e4c8794f19b01d7c1d350 (DEFAULT_TABLE table=test-table [id=928b6247333e4131a036ace9be5bfa46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:22.433306 20061 tablet_service.cc:1467] Processing CreateTablet for tablet 26318310266e4c8794f19b01d7c1d350 (DEFAULT_TABLE table=test-table [id=928b6247333e4131a036ace9be5bfa46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:22.433911 20361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26318310266e4c8794f19b01d7c1d350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.433837 20212 tablet_service.cc:1467] Processing CreateTablet for tablet 26318310266e4c8794f19b01d7c1d350 (DEFAULT_TABLE table=test-table [id=928b6247333e4131a036ace9be5bfa46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:22.434365 20061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26318310266e4c8794f19b01d7c1d350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.434736 20212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26318310266e4c8794f19b01d7c1d350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.440483 20453 tablet_bootstrap.cc:492] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Bootstrap starting.
I20260809 07:48:22.442569 20455 tablet_bootstrap.cc:492] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Bootstrap starting.
I20260809 07:48:22.442770 20453 tablet_bootstrap.cc:654] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.443678 20453 log.cc:824] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:22.443706 20456 tablet_bootstrap.cc:492] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Bootstrap starting.
I20260809 07:48:22.444705 20455 tablet_bootstrap.cc:654] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.445523 20455 log.cc:824] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:22.445928 20453 tablet_bootstrap.cc:492] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: No bootstrap required, opened a new log
I20260809 07:48:22.446157 20453 ts_tablet_manager.cc:1397] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:48:22.446348 20456 tablet_bootstrap.cc:654] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.447003 20456 log.cc:824] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:22.447522 20455 tablet_bootstrap.cc:492] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: No bootstrap required, opened a new log
I20260809 07:48:22.447700 20455 ts_tablet_manager.cc:1397] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 07:48:22.448810 20456 tablet_bootstrap.cc:492] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: No bootstrap required, opened a new log
I20260809 07:48:22.449033 20456 ts_tablet_manager.cc:1397] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 07:48:22.450258 20453 raft_consensus.cc:348] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.450547 20453 raft_consensus.cc:374] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.450616 20453 raft_consensus.cc:729] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e0ebfa3635415b984e927f487e2317, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.451066 20453 consensus_queue.cc:260] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.451568 20455 raft_consensus.cc:348] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.451800 20455 raft_consensus.cc:374] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.451804 20427 heartbeater.cc:503] Master 127.18.71.126:40885 was elected leader, sending a full tablet report...
I20260809 07:48:22.451846 20455 raft_consensus.cc:729] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4454b16ecd7e4edd8354928b7375e7f5, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.452107 20453 ts_tablet_manager.cc:1428] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260809 07:48:22.452324 20455 consensus_queue.cc:260] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.453105 20279 heartbeater.cc:503] Master 127.18.71.126:40885 was elected leader, sending a full tablet report...
I20260809 07:48:22.453341 20455 ts_tablet_manager.cc:1428] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260809 07:48:22.454499 20456 raft_consensus.cc:348] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.454761 20456 raft_consensus.cc:374] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.454844 20456 raft_consensus.cc:729] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fd180c4b7154c15a2e541ddbd6418aa, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.455284 20456 consensus_queue.cc:260] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.455895 20129 heartbeater.cc:503] Master 127.18.71.126:40885 was elected leader, sending a full tablet report...
I20260809 07:48:22.456239 20456 ts_tablet_manager.cc:1428] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:48:22.560725 20460 raft_consensus.cc:482] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:22.560998 20460 raft_consensus.cc:504] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.562150 20460 leader_election.cc:288] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66:40475), 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801)
I20260809 07:48:22.566272 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.562567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:38053 (local address 127.18.71.65:42801)
0809 07:48:22.562693 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.562698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.563123 (+ 425us) server_negotiation.cc:407] Connection header received
0809 07:48:22.563219 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.563221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.563261 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.563345 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:22.563652 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.564129 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.565283 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.565707 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.565838 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.565895 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.566070 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.566145 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 07:48:22.566301 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.562435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:42801 (local address 127.18.71.67:38053)
0809 07:48:22.563014 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.563026 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.563120 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.563369 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.563372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.563389 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:22.563508 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.563513 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.564241 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.564246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.565148 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.565158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.565834 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.565838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.565946 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.565964 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.566054 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.566134 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":136,"spinlock_wait_cycles":1280,"thread_start_us":155,"threads_started":1}
I20260809 07:48:22.566437 20316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.562435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40475 (local address 127.18.71.67:60339)
0809 07:48:22.562577 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.562595 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.562713 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.563035 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.563040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.563064 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:22.563194 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.563203 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.563991 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.563997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.565330 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.565340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.566006 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.566010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.566133 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.566149 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.566243 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.566314 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":19}
I20260809 07:48:22.566437 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.562584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:60339 (local address 127.18.71.66:40475)
0809 07:48:22.562751 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.562756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.562801 (+ 45us) server_negotiation.cc:407] Connection header received
0809 07:48:22.562875 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.562878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.562928 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.562988 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:22.563324 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.563859 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.565466 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.565876 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.566006 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.566060 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.566262 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.566332 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 07:48:22.566901 20081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26318310266e4c8794f19b01d7c1d350" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" is_pre_election: true
I20260809 07:48:22.567011 20232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26318310266e4c8794f19b01d7c1d350" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4454b16ecd7e4edd8354928b7375e7f5" is_pre_election: true
I20260809 07:48:22.567279 20081 raft_consensus.cc:2454] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 0.
I20260809 07:48:22.567329 20232 raft_consensus.cc:2454] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 0.
I20260809 07:48:22.567813 20310 leader_election.cc:302] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4454b16ecd7e4edd8354928b7375e7f5, 96e0ebfa3635415b984e927f487e2317, 9fd180c4b7154c15a2e541ddbd6418aa; no voters:
I20260809 07:48:22.568099 20460 raft_consensus.cc:2781] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:22.568178 20460 raft_consensus.cc:482] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:22.568210 20460 raft_consensus.cc:3037] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.569342 20460 raft_consensus.cc:504] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.569692 20460 leader_election.cc:288] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 election: Requested vote from peers 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66:40475), 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801)
I20260809 07:48:22.570080 20081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26318310266e4c8794f19b01d7c1d350" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fd180c4b7154c15a2e541ddbd6418aa"
I20260809 07:48:22.570075 20232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26318310266e4c8794f19b01d7c1d350" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4454b16ecd7e4edd8354928b7375e7f5"
I20260809 07:48:22.570204 20081 raft_consensus.cc:3037] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.570223 20232 raft_consensus.cc:3037] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.571676 20081 raft_consensus.cc:2454] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 1.
I20260809 07:48:22.572011 20310 leader_election.cc:302] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e0ebfa3635415b984e927f487e2317, 9fd180c4b7154c15a2e541ddbd6418aa; no voters:
I20260809 07:48:22.572108 20232 raft_consensus.cc:2454] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 1.
I20260809 07:48:22.572203 20460 raft_consensus.cc:2781] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:22.572453 20460 raft_consensus.cc:686] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 1 LEADER]: Becoming Leader. State: Replica: 96e0ebfa3635415b984e927f487e2317, State: Running, Role: LEADER
I20260809 07:48:22.572866 20460 consensus_queue.cc:237] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } }
I20260809 07:48:22.576198 19911 catalog_manager.cc:5495] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96e0ebfa3635415b984e927f487e2317 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "96e0ebfa3635415b984e927f487e2317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } health_report { overall_health: UNKNOWN } } }
W20260809 07:48:22.617162 20428 tablet.cc:2369] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:48:22.617236 20280 tablet.cc:2369] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:22.627120 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.623365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.18.71.67:45257)
0809 07:48:22.623495 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.623499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.623514 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:22.623713 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.623716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.623769 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.623857 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:22.624145 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.624759 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.625739 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.625953 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.626031 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.626315 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:22.626762 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:22.626853 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.626957 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.627009 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:48:22.633626 20232 raft_consensus.cc:1261] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Refusing update from remote peer 96e0ebfa3635415b984e927f487e2317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:22.633626 20081 raft_consensus.cc:1261] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Refusing update from remote peer 96e0ebfa3635415b984e927f487e2317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:22.634416 20464 consensus_queue.cc:1035] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:48:22.634784 20130 tablet.cc:2369] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:22.635349 20460 consensus_queue.cc:1035] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:22.643044 20469 mvcc.cc:205] Tried to move back new op lower bound from 7316527933978861568 to 7316527933741608960. Current Snapshot: MvccSnapshot[applied={T|T < 7316527933978861568}]
I20260809 07:48:22.644174 20468 mvcc.cc:205] Tried to move back new op lower bound from 7316527933978861568 to 7316527933741608960. Current Snapshot: MvccSnapshot[applied={T|T < 7316527933978861568}]
I20260809 07:48:22.656075 19912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:22.656332 19912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:22.659243 19912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: Sending DeleteTablet for 3 replicas of tablet 26318310266e4c8794f19b01d7c1d350
I20260809 07:48:22.660177 20212 tablet_service.cc:1514] Processing DeleteTablet for tablet 26318310266e4c8794f19b01d7c1d350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:22 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:47126
I20260809 07:48:22.660296 20361 tablet_service.cc:1514] Processing DeleteTablet for tablet 26318310266e4c8794f19b01d7c1d350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:22 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:60210
I20260809 07:48:22.660831 20061 tablet_service.cc:1514] Processing DeleteTablet for tablet 26318310266e4c8794f19b01d7c1d350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:22 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:56840
I20260809 07:48:22.661067 20481 tablet_replica.cc:330] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: stopping tablet replica
I20260809 07:48:22.661303 20482 tablet_replica.cc:330] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: stopping tablet replica
I20260809 07:48:22.661404 20481 raft_consensus.cc:2229] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:22.661334 19912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 07:48:22.661609 20482 raft_consensus.cc:2229] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 07:48:22.661733 19912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:22.661820 20481 raft_consensus.cc:2258] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:22.661908 20482 raft_consensus.cc:2258] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:22.662209 20480 tablet_replica.cc:330] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: stopping tablet replica
I20260809 07:48:22.663470 20482 ts_tablet_manager.cc:1905] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:22.666489 20482 ts_tablet_manager.cc:1918] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:22.666611 20482 log.cc:1196] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/26318310266e4c8794f19b01d7c1d350
I20260809 07:48:22.666877 20061 tablet_service.cc:1467] Processing CreateTablet for tablet 4eefb417e9cf45f286e344d49eff1894 (DEFAULT_TABLE table=test-table [id=634cf81cf4344bfcab91f17f66908ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:22.667119 20061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eefb417e9cf45f286e344d49eff1894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.667188 20361 tablet_service.cc:1467] Processing CreateTablet for tablet 4eefb417e9cf45f286e344d49eff1894 (DEFAULT_TABLE table=test-table [id=634cf81cf4344bfcab91f17f66908ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:22.667475 20361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eefb417e9cf45f286e344d49eff1894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.669059 20212 tablet_service.cc:1467] Processing CreateTablet for tablet 4eefb417e9cf45f286e344d49eff1894 (DEFAULT_TABLE table=test-table [id=634cf81cf4344bfcab91f17f66908ee7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:22.669245 20456 tablet_bootstrap.cc:492] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: Bootstrap starting.
I20260809 07:48:22.669354 20212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eefb417e9cf45f286e344d49eff1894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.669813 20482 ts_tablet_manager.cc:1939] T 26318310266e4c8794f19b01d7c1d350 P 9fd180c4b7154c15a2e541ddbd6418aa: Deleting consensus metadata
I20260809 07:48:22.670398 20456 tablet_bootstrap.cc:654] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.665732 20480 raft_consensus.cc:2229] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:22.670971 20481 ts_tablet_manager.cc:1905] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:22.671396 20480 raft_consensus.cc:2258] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:22.671393 19900 catalog_manager.cc:4841] TS 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801): tablet 26318310266e4c8794f19b01d7c1d350 (table test-table [id=928b6247333e4131a036ace9be5bfa46]) successfully deleted
I20260809 07:48:22.671509 20453 tablet_bootstrap.cc:492] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: Bootstrap starting.
I20260809 07:48:22.672293 20455 tablet_bootstrap.cc:492] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: Bootstrap starting.
I20260809 07:48:22.672518 20453 tablet_bootstrap.cc:654] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.673290 20455 tablet_bootstrap.cc:654] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.673687 20453 tablet_bootstrap.cc:492] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: No bootstrap required, opened a new log
I20260809 07:48:22.673774 20453 ts_tablet_manager.cc:1397] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 07:48:22.674223 20453 raft_consensus.cc:348] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.674335 20453 raft_consensus.cc:374] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.674365 20453 raft_consensus.cc:729] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e0ebfa3635415b984e927f487e2317, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.674486 20453 consensus_queue.cc:260] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.674727 20453 ts_tablet_manager.cc:1428] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 07:48:22.675585 20481 ts_tablet_manager.cc:1918] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:22.675685 20481 log.cc:1196] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/26318310266e4c8794f19b01d7c1d350
I20260809 07:48:22.676048 20456 tablet_bootstrap.cc:492] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: No bootstrap required, opened a new log
I20260809 07:48:22.676115 20481 ts_tablet_manager.cc:1939] T 26318310266e4c8794f19b01d7c1d350 P 96e0ebfa3635415b984e927f487e2317: Deleting consensus metadata
I20260809 07:48:22.676133 20456 ts_tablet_manager.cc:1397] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 07:48:22.676594 20456 raft_consensus.cc:348] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.676707 20456 raft_consensus.cc:374] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.676748 20456 raft_consensus.cc:729] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fd180c4b7154c15a2e541ddbd6418aa, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.676857 20456 consensus_queue.cc:260] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.677096 20456 ts_tablet_manager.cc:1428] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:22.677259 19900 catalog_manager.cc:4841] TS 96e0ebfa3635415b984e927f487e2317 (127.18.71.67:45257): tablet 26318310266e4c8794f19b01d7c1d350 (table test-table [id=928b6247333e4131a036ace9be5bfa46]) successfully deleted
I20260809 07:48:22.678627 20480 ts_tablet_manager.cc:1905] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:22.679474 20455 tablet_bootstrap.cc:492] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: No bootstrap required, opened a new log
I20260809 07:48:22.679579 20455 ts_tablet_manager.cc:1397] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 07:48:22.680030 20455 raft_consensus.cc:348] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.680155 20455 raft_consensus.cc:374] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.680187 20455 raft_consensus.cc:729] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4454b16ecd7e4edd8354928b7375e7f5, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.680311 20455 consensus_queue.cc:260] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.680586 20455 ts_tablet_manager.cc:1428] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:22.681169 20480 ts_tablet_manager.cc:1918] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:22.681267 20480 log.cc:1196] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/26318310266e4c8794f19b01d7c1d350
I20260809 07:48:22.681566 20480 ts_tablet_manager.cc:1939] T 26318310266e4c8794f19b01d7c1d350 P 4454b16ecd7e4edd8354928b7375e7f5: Deleting consensus metadata
I20260809 07:48:22.682898 19896 catalog_manager.cc:4841] TS 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66:40475): tablet 26318310266e4c8794f19b01d7c1d350 (table test-table [id=928b6247333e4131a036ace9be5bfa46]) successfully deleted
I20260809 07:48:22.721619 20476 raft_consensus.cc:482] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:22.721779 20476 raft_consensus.cc:504] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.722263 20476 leader_election.cc:288] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801), 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66:40475)
I20260809 07:48:22.722690 20232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eefb417e9cf45f286e344d49eff1894" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4454b16ecd7e4edd8354928b7375e7f5" is_pre_election: true
I20260809 07:48:22.722769 20081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eefb417e9cf45f286e344d49eff1894" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" is_pre_election: true
I20260809 07:48:22.722878 20232 raft_consensus.cc:2454] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 0.
I20260809 07:48:22.722927 20081 raft_consensus.cc:2454] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 0.
I20260809 07:48:22.723237 20306 leader_election.cc:302] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4454b16ecd7e4edd8354928b7375e7f5, 96e0ebfa3635415b984e927f487e2317; no voters:
I20260809 07:48:22.723507 20476 raft_consensus.cc:2781] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:22.723613 20476 raft_consensus.cc:482] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:22.723671 20476 raft_consensus.cc:3037] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.724762 20476 raft_consensus.cc:504] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.725134 20476 leader_election.cc:288] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 election: Requested vote from peers 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801), 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66:40475)
I20260809 07:48:22.725492 20081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eefb417e9cf45f286e344d49eff1894" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fd180c4b7154c15a2e541ddbd6418aa"
I20260809 07:48:22.725497 20232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eefb417e9cf45f286e344d49eff1894" candidate_uuid: "96e0ebfa3635415b984e927f487e2317" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4454b16ecd7e4edd8354928b7375e7f5"
I20260809 07:48:22.725608 20232 raft_consensus.cc:3037] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.725633 20081 raft_consensus.cc:3037] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.726577 20081 raft_consensus.cc:2454] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 1.
I20260809 07:48:22.726768 20232 raft_consensus.cc:2454] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e0ebfa3635415b984e927f487e2317 in term 1.
I20260809 07:48:22.726890 20310 leader_election.cc:302] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e0ebfa3635415b984e927f487e2317, 9fd180c4b7154c15a2e541ddbd6418aa; no voters:
I20260809 07:48:22.727089 20476 raft_consensus.cc:2781] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:22.727222 20476 raft_consensus.cc:686] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 1 LEADER]: Becoming Leader. State: Replica: 96e0ebfa3635415b984e927f487e2317, State: Running, Role: LEADER
I20260809 07:48:22.727363 20476 consensus_queue.cc:237] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.729101 19912 catalog_manager.cc:5495] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96e0ebfa3635415b984e927f487e2317 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "96e0ebfa3635415b984e927f487e2317" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } health_report { overall_health: HEALTHY } } }
Aug 09 07:48:22 dist-test-slave-10pc krb5kdc[19864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261701, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:22.767437 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.758230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40885 (local address 127.0.0.1:52808)
0809 07:48:22.758528 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.758544 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.758668 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.758956 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.758958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.759332 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:22.759429 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.759434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.760560 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.760568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.761116 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.761123 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.761765 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.761767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.761830 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.762433 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:22.762445 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:22.763885 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:22.765856 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.765865 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.765870 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.766102 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.766392 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:22.766394 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:22.766396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:22.766471 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:22.766990 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:22.766996 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:22.767096 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.767185 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.767299 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":106,"threads_started":1}
I20260809 07:48:22.767594 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.758334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.18.71.126:40885)
0809 07:48:22.758488 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.758493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.758634 (+ 141us) server_negotiation.cc:407] Connection header received
0809 07:48:22.758763 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.758766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.758821 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.758925 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:22.759551 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.760405 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.761233 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.761646 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.761758 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.764040 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:22.764044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:22.764047 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:22.764071 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:22.765738 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.766203 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.766206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.766208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.766297 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:22.766567 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:22.766570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:22.766572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:22.766878 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:22.766990 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.767276 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.767401 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:48:22.771689 19912 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:52808:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 07:48:22.772524 19912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:22.777685 20212 tablet_service.cc:1467] Processing CreateTablet for tablet 89fc5e8a9450458cb0b369f154319894 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=375f0c1c23184c01bf344201e81b25f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:22.777755 20061 tablet_service.cc:1467] Processing CreateTablet for tablet 89fc5e8a9450458cb0b369f154319894 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=375f0c1c23184c01bf344201e81b25f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:22.778180 20212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89fc5e8a9450458cb0b369f154319894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.778175 20061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89fc5e8a9450458cb0b369f154319894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.780795 20361 tablet_service.cc:1467] Processing CreateTablet for tablet 89fc5e8a9450458cb0b369f154319894 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=375f0c1c23184c01bf344201e81b25f2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:22.781070 20361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89fc5e8a9450458cb0b369f154319894. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:22.783416 20455 tablet_bootstrap.cc:492] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: Bootstrap starting.
I20260809 07:48:22.783560 20453 tablet_bootstrap.cc:492] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: Bootstrap starting.
I20260809 07:48:22.784626 20455 tablet_bootstrap.cc:654] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.784847 20456 tablet_bootstrap.cc:492] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: Bootstrap starting.
I20260809 07:48:22.785071 20453 tablet_bootstrap.cc:654] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.785899 20456 tablet_bootstrap.cc:654] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:22.786262 20455 tablet_bootstrap.cc:492] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: No bootstrap required, opened a new log
I20260809 07:48:22.786273 20453 tablet_bootstrap.cc:492] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: No bootstrap required, opened a new log
I20260809 07:48:22.786341 20453 ts_tablet_manager.cc:1397] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:22.786341 20455 ts_tablet_manager.cc:1397] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 07:48:22.786775 20453 raft_consensus.cc:348] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.786773 20455 raft_consensus.cc:348] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.786895 20455 raft_consensus.cc:374] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.786895 20453 raft_consensus.cc:374] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.786931 20455 raft_consensus.cc:729] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4454b16ecd7e4edd8354928b7375e7f5, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.786931 20453 raft_consensus.cc:729] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e0ebfa3635415b984e927f487e2317, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.787060 20453 consensus_queue.cc:260] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.787060 20455 consensus_queue.cc:260] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.787334 20453 ts_tablet_manager.cc:1428] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:22.787334 20455 ts_tablet_manager.cc:1428] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 07:48:22.787631 20476 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.787773 20476 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:22.787967 20461 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.788072 20461 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:22.788782 20456 tablet_bootstrap.cc:492] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: No bootstrap required, opened a new log
I20260809 07:48:22.788866 20456 ts_tablet_manager.cc:1397] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:48:22.789309 20456 raft_consensus.cc:348] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.789418 20456 raft_consensus.cc:374] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:22.789453 20456 raft_consensus.cc:729] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fd180c4b7154c15a2e541ddbd6418aa, State: Initialized, Role: FOLLOWER
I20260809 07:48:22.789582 20456 consensus_queue.cc:260] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.789865 20462 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.790026 20462 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:22.789893 20456 ts_tablet_manager.cc:1428] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:22.911863 20461 raft_consensus.cc:482] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:22.912056 20461 raft_consensus.cc:504] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.913213 20461 leader_election.cc:288] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801), 96e0ebfa3635415b984e927f487e2317 (127.18.71.67:45257)
I20260809 07:48:22.918485 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.914088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:56261 (local address 127.18.71.67:45257)
0809 07:48:22.914230 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.914237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.914307 (+ 70us) server_negotiation.cc:407] Connection header received
0809 07:48:22.914436 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.914438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.914476 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.914535 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:22.914848 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.915528 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.917218 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.917835 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.917949 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.918001 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.918284 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.918358 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 07:48:22.918485 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.913594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:57933 (local address 127.18.71.65:42801)
0809 07:48:22.913733 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.913738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.913952 (+ 214us) server_negotiation.cc:407] Connection header received
0809 07:48:22.914131 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.914134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.914191 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.914285 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:22.914641 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.915415 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.917218 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.917819 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.917949 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.918001 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.918284 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.918358 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:48:22.918556 20497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.913486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:42801 (local address 127.18.71.66:57933)
0809 07:48:22.913841 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.913859 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.913986 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.914314 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.914317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.914343 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:22.914495 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.914501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.915518 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.915521 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.917081 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.917093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.917953 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.917956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.918110 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.918168 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.918249 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.918358 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":191,"spinlock_wait_cycles":7296,"thread_start_us":109,"threads_started":1}
I20260809 07:48:22.918602 20498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.913924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:45257 (local address 127.18.71.66:56261)
0809 07:48:22.914216 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.914227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.914341 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.914583 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.914589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.914624 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:22.914725 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.914729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.915626 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.915629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.917081 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.917093 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.917953 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.917956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.918104 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.918168 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.918249 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.918358 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":98,"spinlock_wait_cycles":384,"thread_start_us":110,"threads_started":1}
I20260809 07:48:22.919148 20081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fc5e8a9450458cb0b369f154319894" candidate_uuid: "4454b16ecd7e4edd8354928b7375e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" is_pre_election: true
I20260809 07:48:22.919193 20381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fc5e8a9450458cb0b369f154319894" candidate_uuid: "4454b16ecd7e4edd8354928b7375e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e0ebfa3635415b984e927f487e2317" is_pre_election: true
I20260809 07:48:22.919301 20081 raft_consensus.cc:2454] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4454b16ecd7e4edd8354928b7375e7f5 in term 0.
I20260809 07:48:22.919466 20381 raft_consensus.cc:2454] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4454b16ecd7e4edd8354928b7375e7f5 in term 0.
I20260809 07:48:22.919699 20158 leader_election.cc:302] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4454b16ecd7e4edd8354928b7375e7f5, 9fd180c4b7154c15a2e541ddbd6418aa; no voters:
I20260809 07:48:22.920027 20461 raft_consensus.cc:2781] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:22.920102 20461 raft_consensus.cc:482] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:22.920151 20461 raft_consensus.cc:3037] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.921252 20461 raft_consensus.cc:504] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.921568 20461 leader_election.cc:288] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801), 96e0ebfa3635415b984e927f487e2317 (127.18.71.67:45257)
I20260809 07:48:22.921929 20081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fc5e8a9450458cb0b369f154319894" candidate_uuid: "4454b16ecd7e4edd8354928b7375e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fd180c4b7154c15a2e541ddbd6418aa"
I20260809 07:48:22.921978 20381 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89fc5e8a9450458cb0b369f154319894" candidate_uuid: "4454b16ecd7e4edd8354928b7375e7f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e0ebfa3635415b984e927f487e2317"
I20260809 07:48:22.922078 20081 raft_consensus.cc:3037] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.922101 20381 raft_consensus.cc:3037] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:22.922928 20081 raft_consensus.cc:2454] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4454b16ecd7e4edd8354928b7375e7f5 in term 1.
I20260809 07:48:22.923112 20381 raft_consensus.cc:2454] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4454b16ecd7e4edd8354928b7375e7f5 in term 1.
I20260809 07:48:22.923259 20158 leader_election.cc:302] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4454b16ecd7e4edd8354928b7375e7f5, 9fd180c4b7154c15a2e541ddbd6418aa; no voters:
I20260809 07:48:22.923451 20461 raft_consensus.cc:2781] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:22.923688 20461 raft_consensus.cc:686] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4454b16ecd7e4edd8354928b7375e7f5, State: Running, Role: LEADER
I20260809 07:48:22.923944 20461 consensus_queue.cc:237] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } }
I20260809 07:48:22.925405 20499 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: TxnStatusTablet state changed. Reason: New leader 4454b16ecd7e4edd8354928b7375e7f5. Latest consensus state: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.925514 20499 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:22.926451 20501 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 07:48:22.926652 20501 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 07:48:22.926553 19912 catalog_manager.cc:5495] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } health_report { overall_health: UNKNOWN } } }
I20260809 07:48:22.940977 20493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.937289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40475 (local address 127.0.0.1:47130)
0809 07:48:22.937452 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.937468 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.937594 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.937922 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.937924 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.937931 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:22.938039 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.938044 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.938766 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.938769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.939367 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.939375 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.939821 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.939825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.939928 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.939967 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:22.940684 (+ 717us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:22.940701 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.940780 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.940865 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 07:48:22.940974 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.937373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.18.71.66:40475)
0809 07:48:22.937495 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.937502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.937516 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:48:22.937737 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.937740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.937798 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.937895 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:22.938165 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.938657 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.939493 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.939712 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.939819 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.940123 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:22.940590 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:22.940712 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.940791 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.940840 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:48:22.943832 20081 raft_consensus.cc:1261] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Refusing update from remote peer 4454b16ecd7e4edd8354928b7375e7f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:22.944053 20381 raft_consensus.cc:1261] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Refusing update from remote peer 4454b16ecd7e4edd8354928b7375e7f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:22.944334 20499 consensus_queue.cc:1035] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:22.944552 20461 consensus_queue.cc:1035] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:22.946156 20462 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: TxnStatusTablet state changed. Reason: New leader 4454b16ecd7e4edd8354928b7375e7f5. Latest consensus state: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.946283 20462 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:22.946250 20476 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: TxnStatusTablet state changed. Reason: New leader 4454b16ecd7e4edd8354928b7375e7f5. Latest consensus state: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.946375 20476 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:22.948181 20499 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.948285 20499 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:22.949236 20462 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.949343 20462 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 9fd180c4b7154c15a2e541ddbd6418aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:22.950469 20461 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.950567 20461 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 4454b16ecd7e4edd8354928b7375e7f5: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:22.952536 20476 tablet_replica.cc:439] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4454b16ecd7e4edd8354928b7375e7f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 } } peers { permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 } } peers { permanent_uuid: "96e0ebfa3635415b984e927f487e2317" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45257 } } }
I20260809 07:48:22.952662 20476 tablet_replica.cc:442] T 89fc5e8a9450458cb0b369f154319894 P 96e0ebfa3635415b984e927f487e2317: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:22.964331 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.960781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40475 (local address 127.18.71.67:40777)
0809 07:48:22.961129 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.961141 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.961225 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.961443 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.961445 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.961452 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:22.961543 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.961548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.962509 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.962511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.963137 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.963146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.963698 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.963700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.963763 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.963804 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:22.964083 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:22.964087 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.964134 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.964203 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":84,"threads_started":1}
I20260809 07:48:22.964339 20448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.960852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:40777 (local address 127.18.71.66:40475)
0809 07:48:22.960966 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.960972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.961211 (+ 239us) server_negotiation.cc:407] Connection header received
0809 07:48:22.961297 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.961300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.961348 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.961427 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:22.961643 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.962410 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.963252 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.963591 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.963684 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.963920 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:22.963999 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:22.964069 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.964144 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.964197 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 07:48:22.971093 20081 raft_consensus.cc:1261] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Refusing update from remote peer 96e0ebfa3635415b984e927f487e2317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:22.971204 20232 raft_consensus.cc:1261] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Refusing update from remote peer 96e0ebfa3635415b984e927f487e2317: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:22.971540 20476 consensus_queue.cc:1035] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fd180c4b7154c15a2e541ddbd6418aa" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 42801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:22.971731 20472 consensus_queue.cc:1035] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4454b16ecd7e4edd8354928b7375e7f5" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40475 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:22.977860 20467 mvcc.cc:205] Tried to move back new op lower bound from 7316527935365824512 to 7316527934372638720. Current Snapshot: MvccSnapshot[applied={T|T < 7316527935365824512}]
I20260809 07:48:22.997025 20451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.992913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:37797 (local address 127.18.71.67:45257)
0809 07:48:22.993054 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:22.993059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:22.993317 (+ 258us) server_negotiation.cc:407] Connection header received
0809 07:48:22.993522 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:22.993525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:22.993576 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:22.993665 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:22.994125 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.994666 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.995909 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.996123 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.996203 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.996549 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:22.996623 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:22.996678 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:22.996828 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 07:48:22.996893 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 07:48:22.997035 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:22.992862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:45257 (local address 127.18.71.66:37797)
0809 07:48:22.993237 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:22.993252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:22.993392 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:22.993841 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:22.993858 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:22.993866 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:22.994011 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.994017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.994774 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.994778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.995791 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:22.995802 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:22.996234 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:22.996237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:22.996335 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:22.996389 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:22.996708 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:22.996714 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 07:48:22.996784 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 07:48:22.996862 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":134,"threads_started":1}
I20260809 07:48:23.016505 19911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:23.016688 19911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:23.018731 19911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27178697edc9420597ecd386988dc5b7: Sending DeleteTablet for 3 replicas of tablet 4eefb417e9cf45f286e344d49eff1894
I20260809 07:48:23.019332 20212 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eefb417e9cf45f286e344d49eff1894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:23 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:47126
I20260809 07:48:23.019344 20061 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eefb417e9cf45f286e344d49eff1894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:23 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:56840
I20260809 07:48:23.019526 20482 tablet_replica.cc:330] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: stopping tablet replica
I20260809 07:48:23.019529 20480 tablet_replica.cc:330] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: stopping tablet replica
I20260809 07:48:23.019647 20482 raft_consensus.cc:2229] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:23.019649 20480 raft_consensus.cc:2229] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:23.019768 20480 raft_consensus.cc:2258] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:23.019786 20482 raft_consensus.cc:2258] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:23.020610 20482 ts_tablet_manager.cc:1905] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:23.020661 20480 ts_tablet_manager.cc:1905] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:23.021447 20361 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eefb417e9cf45f286e344d49eff1894 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:23 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:60210
I20260809 07:48:23.023540 20482 ts_tablet_manager.cc:1918] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:23.023538 20480 ts_tablet_manager.cc:1918] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:23.023615 20482 log.cc:1196] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/4eefb417e9cf45f286e344d49eff1894
I20260809 07:48:23.023622 20480 log.cc:1196] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/4eefb417e9cf45f286e344d49eff1894
I20260809 07:48:23.023891 20482 ts_tablet_manager.cc:1939] T 4eefb417e9cf45f286e344d49eff1894 P 9fd180c4b7154c15a2e541ddbd6418aa: Deleting consensus metadata
I20260809 07:48:23.023954 20481 tablet_replica.cc:330] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: stopping tablet replica
I20260809 07:48:23.024102 20481 raft_consensus.cc:2229] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:23.024307 20481 raft_consensus.cc:2258] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:23.024721 19900 catalog_manager.cc:4841] TS 9fd180c4b7154c15a2e541ddbd6418aa (127.18.71.65:42801): tablet 4eefb417e9cf45f286e344d49eff1894 (table test-table [id=634cf81cf4344bfcab91f17f66908ee7]) successfully deleted
I20260809 07:48:23.024816 20480 ts_tablet_manager.cc:1939] T 4eefb417e9cf45f286e344d49eff1894 P 4454b16ecd7e4edd8354928b7375e7f5: Deleting consensus metadata
I20260809 07:48:23.025090 20481 ts_tablet_manager.cc:1905] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:23.025482 19896 catalog_manager.cc:4841] TS 4454b16ecd7e4edd8354928b7375e7f5 (127.18.71.66:40475): tablet 4eefb417e9cf45f286e344d49eff1894 (table test-table [id=634cf81cf4344bfcab91f17f66908ee7]) successfully deleted
I20260809 07:48:23.027738 20481 ts_tablet_manager.cc:1918] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:23.027817 20481 log.cc:1196] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/4eefb417e9cf45f286e344d49eff1894
I20260809 07:48:23.028110 20481 ts_tablet_manager.cc:1939] T 4eefb417e9cf45f286e344d49eff1894 P 96e0ebfa3635415b984e927f487e2317: Deleting consensus metadata
I20260809 07:48:23.028899 19900 catalog_manager.cc:4841] TS 96e0ebfa3635415b984e927f487e2317 (127.18.71.67:45257): tablet 4eefb417e9cf45f286e344d49eff1894 (table test-table [id=634cf81cf4344bfcab91f17f66908ee7]) successfully deleted
Aug 09 07:48:23 dist-test-slave-10pc krb5kdc[19864](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261702, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:23.043797 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:23.021893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.18.71.65:42801)
0809 07:48:23.022002 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:23.022007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:23.032823 (+ 10816us) server_negotiation.cc:407] Connection header received
0809 07:48:23.032973 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:23.032976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:23.033028 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:23.033101 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:23.033990 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:23.034607 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:23.035770 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:23.035981 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:23.036063 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:23.039855 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:23.039870 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:23.039888 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:23.039915 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:23.041951 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:23.042405 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:23.042412 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:23.042414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:23.042529 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:23.042792 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:23.042795 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:23.042797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:23.043117 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:23.043343 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:23.043474 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 07:48:23.043601 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 07:48:23.044387 20123 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56846
I20260809 07:48:23.054180 19965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:23.046329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52816 (local address 127.18.71.126:40885)
0809 07:48:23.046467 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:23.046472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:23.046490 (+ 18us) server_negotiation.cc:407] Connection header received
0809 07:48:23.046594 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:23.046598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:23.046649 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:23.046739 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:23.047347 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:23.048145 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:23.049030 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:23.049391 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:23.049498 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:23.050839 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:23.050843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:23.050846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:23.050875 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:23.052523 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:23.052921 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:23.052925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:23.052927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:23.053022 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:23.053281 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:23.053283 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:23.053285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:23.053506 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:23.053636 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:23.053867 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 07:48:23.053999 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260809 07:48:23.054659 19911 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52816
I20260809 07:48:23.056147 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 19990
I20260809 07:48:23.068850 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 20138
I20260809 07:48:23.078300 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 20289
I20260809 07:48:23.087016 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 19878
2026-08-09T07:48:23Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2877 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:23 dist-test-slave-10pc krb5kdc[20523](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:23 dist-test-slave-10pc krb5kdc[20523](info): set up 2 sockets
Aug 09 07:48:23 dist-test-slave-10pc krb5kdc[20523](info): commencing operation
krb5kdc: starting...
W20260809 07:48:24.160393 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:24 dist-test-slave-10pc krb5kdc[20523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:24.291092 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42059
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:42059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:24.404695 20536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:24.404970 20536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:24.405012 20536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:24.409082 20536 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:24.409137 20536 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 07:48:24.409154 20536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:24.409173 20536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:24.409236 20536 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:24.409264 20536 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:24.414530 20536 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:42059
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:24.415583 20536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:24.416741 20536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:24.425418 20541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:24.425446 20544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:24.425419 20542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:24.425721 20536 server_base.cc:1034] running on GCE node
I20260809 07:48:24.426339 20536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:24.427557 20536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:24.428725 20536 hybrid_clock.cc:648] HybridClock initialized: now 1786261704428717 us; error 32 us; skew 500 ppm
Aug 09 07:48:24 dist-test-slave-10pc krb5kdc[20523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:24.431434 20536 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:24.432358 20536 webserver.cc:460] Webserver started at http://127.18.71.126:36311/ using document root <none> and password file <none>
I20260809 07:48:24.432966 20536 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:24.433028 20536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:24.433198 20536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:24.435245 20536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "601cb7bd3cdc4affb6ad824debceba79"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "f94a068d24c00fdb2b07e154d0d670a5"
server_key_iv: "76b8d4654712237d73bed961dfd32f06"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.435760 20536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "601cb7bd3cdc4affb6ad824debceba79"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "f94a068d24c00fdb2b07e154d0d670a5"
server_key_iv: "76b8d4654712237d73bed961dfd32f06"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.439419 20536 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:48:24.441886 20551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:24.443166 20536 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:24.443308 20536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "601cb7bd3cdc4affb6ad824debceba79"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "f94a068d24c00fdb2b07e154d0d670a5"
server_key_iv: "76b8d4654712237d73bed961dfd32f06"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.443429 20536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:24.467106 20536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:24.470243 20536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:24.470399 20536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:24.478245 20536 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:42059
I20260809 07:48:24.478345 20618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:42059 every 8 connection(s)
I20260809 07:48:24.479213 20536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:24.482049 20619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:24.487608 20619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Bootstrap starting.
I20260809 07:48:24.487823 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 20536
I20260809 07:48:24.487951 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:24.488206 18717 external_mini_cluster.cc:1428] Setting key d3602ca70eea25f1012dcb7efafc5a8f
I20260809 07:48:24.490341 20619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:24.491101 20619 log.cc:824] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:24.493242 20619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: No bootstrap required, opened a new log
I20260809 07:48:24.496773 20619 raft_consensus.cc:348] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "601cb7bd3cdc4affb6ad824debceba79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42059 } }
I20260809 07:48:24.497115 20619 raft_consensus.cc:374] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:24.497184 20619 raft_consensus.cc:729] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 601cb7bd3cdc4affb6ad824debceba79, State: Initialized, Role: FOLLOWER
I20260809 07:48:24.497639 20619 consensus_queue.cc:260] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "601cb7bd3cdc4affb6ad824debceba79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42059 } }
I20260809 07:48:24.497766 20619 raft_consensus.cc:388] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:24.497851 20619 raft_consensus.cc:482] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 09 07:48:24 dist-test-slave-10pc krb5kdc[20523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:24.497956 20619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:24.498968 20619 raft_consensus.cc:504] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "601cb7bd3cdc4affb6ad824debceba79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42059 } }
I20260809 07:48:24.499325 20619 leader_election.cc:302] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 601cb7bd3cdc4affb6ad824debceba79; no voters:
I20260809 07:48:24.499624 20619 leader_election.cc:288] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:24.499734 20625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:24.499955 20625 raft_consensus.cc:686] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [term 1 LEADER]: Becoming Leader. State: Replica: 601cb7bd3cdc4affb6ad824debceba79, State: Running, Role: LEADER
I20260809 07:48:24.500277 20625 consensus_queue.cc:237] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "601cb7bd3cdc4affb6ad824debceba79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42059 } }
I20260809 07:48:24.501025 20619 sys_catalog.cc:564] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:24.501876 20626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "601cb7bd3cdc4affb6ad824debceba79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "601cb7bd3cdc4affb6ad824debceba79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42059 } } }
I20260809 07:48:24.502219 20627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 601cb7bd3cdc4affb6ad824debceba79. Latest consensus state: current_term: 1 leader_uuid: "601cb7bd3cdc4affb6ad824debceba79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "601cb7bd3cdc4affb6ad824debceba79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42059 } } }
I20260809 07:48:24.502355 20627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:24.502420 20626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:24.504667 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:24.489626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37594 (local address 127.18.71.126:42059)
0809 07:48:24.490038 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:24.490054 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:24.490089 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:48:24.490778 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:24.490797 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:24.491114 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:24.491435 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:24.492259 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.493281 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.494403 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.495449 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.495570 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:24.498547 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:24.498553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:24.498565 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:24.498590 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:24.500686 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.501200 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.501206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.501212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.501378 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.501714 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.501718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.501720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.502296 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:24.502484 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:24.503395 (+ 911us) server_negotiation.cc:299] Negotiation successful
0809 07:48:24.503724 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":146,"threads_started":1}
W20260809 07:48:24.505996 20641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:24.506073 20641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:24.506134 20635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:24.506975 20635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:24.511860 20635 catalog_manager.cc:1347] Generated new cluster ID: 6efdedc3457d4ae2b2582fa86ad7b633
I20260809 07:48:24.511927 20635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:24.543550 20635 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:24.544936 20635 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:24.551570 20635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Generated new TSK 0
I20260809 07:48:24.552404 20635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:24.588112 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42059
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:24.703091 20648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:24.703464 20648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:24.703547 20648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:24.707656 20648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:24.707748 20648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:24.707819 20648 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 07:48:24.707870 20648 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:24.707932 20648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:24.713436 20648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:42059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:24.714560 20648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:24.715629 20648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:48:24.725476 20648 server_base.cc:1034] running on GCE node
W20260809 07:48:24.725426 20653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:24.725485 20654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:24.725426 20656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:24.726050 20648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:24.726680 20648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:24.727877 20648 hybrid_clock.cc:648] HybridClock initialized: now 1786261704727839 us; error 50 us; skew 500 ppm
Aug 09 07:48:24 dist-test-slave-10pc krb5kdc[20523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:24.730834 20648 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:24.731673 20648 webserver.cc:460] Webserver started at http://127.18.71.65:39091/ using document root <none> and password file <none>
I20260809 07:48:24.732268 20648 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:24.732367 20648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:24.732630 20648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:24.734625 20648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "37da323536664cf2af59ba85eb42d4eb"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "0e153dadd7b25f13476c2f9a63baab20"
server_key_iv: "6d497511d189a03835560def1b8c5f0f"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.735284 20648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "37da323536664cf2af59ba85eb42d4eb"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "0e153dadd7b25f13476c2f9a63baab20"
server_key_iv: "6d497511d189a03835560def1b8c5f0f"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.738971 20648 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:24.741420 20663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:24.742583 20648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:24.742729 20648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "37da323536664cf2af59ba85eb42d4eb"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "0e153dadd7b25f13476c2f9a63baab20"
server_key_iv: "6d497511d189a03835560def1b8c5f0f"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.742842 20648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:24.759874 20648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:24.763152 20648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:24.763368 20648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:24.764905 20648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:24.764979 20648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:24.765055 20648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:24.765100 20648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:24 dist-test-slave-10pc krb5kdc[20523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:24.775835 20648 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:35625
I20260809 07:48:24.775907 20786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:35625 every 8 connection(s)
I20260809 07:48:24.777032 20648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:24.779474 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:24.765678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:50417 (local address 127.18.71.126:42059)
0809 07:48:24.765849 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:24.765854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:24.767174 (+ 1320us) server_negotiation.cc:407] Connection header received
0809 07:48:24.768579 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:24.768582 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:24.768630 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:24.768706 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:24.770404 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.770929 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.772755 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.772988 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.773072 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:24.775788 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:24.775792 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:24.775795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:24.775825 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:24.777366 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.777975 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.777979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.777981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.778080 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.778394 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.778396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.778397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.778578 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:24.778665 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:24.779250 (+ 585us) server_negotiation.cc:299] Negotiation successful
0809 07:48:24.779353 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 07:48:24.780371 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:24.766498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42059 (local address 127.18.71.65:50417)
0809 07:48:24.766997 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:24.767056 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:24.768332 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:24.768939 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:24.768950 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:24.769471 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:24.770222 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:24.770246 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.771054 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.771059 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:24.772628 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:24.772638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.773159 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.773163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:24.773376 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:24.774038 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:24.774060 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:24.775617 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:24.777484 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:24.777490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:24.777506 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:24.777863 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:24.778172 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:24.778174 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:24.778176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:24.778310 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:24.778706 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:24.778712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:24.779014 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 07:48:24.779205 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 07:48:24.779479 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":110,"threads_started":1}
I20260809 07:48:24.781819 20788 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42059
I20260809 07:48:24.781980 20788 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:24.782466 20788 heartbeater.cc:511] Master 127.18.71.126:42059 requested a full tablet report, sending...
I20260809 07:48:24.784150 20569 ts_manager.cc:194] Registered new tserver with Master: 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625)
I20260809 07:48:24.785341 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 20648
I20260809 07:48:24.785470 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:24.785672 20569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:50417
I20260809 07:48:24.785778 18717 external_mini_cluster.cc:1428] Setting key 243f1787fd9875396d4605b04990810a
I20260809 07:48:24.791894 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:24.784046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:38753 (local address 127.18.71.126:42059)
0809 07:48:24.784168 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:24.784171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:24.784323 (+ 152us) server_negotiation.cc:407] Connection header received
0809 07:48:24.784547 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:24.784549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:24.784589 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:24.784656 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:24.785180 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.785938 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.786966 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.787269 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.787374 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:24.788603 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:24.788606 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:24.788609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:24.788635 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:24.790233 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.790649 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.790654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.790658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.790759 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.791029 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.791032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.791034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.791244 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:24.791351 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:24.791630 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 07:48:24.791738 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:48:24.791905 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:24.783916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42059 (local address 127.18.71.65:38753)
0809 07:48:24.784239 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:24.784256 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:24.784388 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:24.784701 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:24.784705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:24.784910 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:24.785054 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:24.785061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.786084 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.786087 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:24.786832 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:24.786841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.787398 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.787402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:24.787499 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:24.788055 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:24.788070 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:24.788414 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:24.790358 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:24.790361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:24.790362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:24.790554 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:24.790854 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:24.790856 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:24.790857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:24.790931 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:24.791358 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:24.791364 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:24.791460 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 07:48:24.791574 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 07:48:24.791715 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:24.816161 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42059
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:24.928651 20799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:24.928933 20799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:24.929010 20799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:24.933013 20799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:24.933111 20799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:24.933190 20799 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 07:48:24.933234 20799 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:24.933310 20799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:24.938768 20799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:42059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:24.939924 20799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:24.941045 20799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:48:24.950955 20799 server_base.cc:1034] running on GCE node
W20260809 07:48:24.951015 20805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:24.950970 20804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:24.950939 20807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:24.951623 20799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:24.952365 20799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:24.953558 20799 hybrid_clock.cc:648] HybridClock initialized: now 1786261704953541 us; error 31 us; skew 500 ppm
Aug 09 07:48:24 dist-test-slave-10pc krb5kdc[20523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:24.956313 20799 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:24.957238 20799 webserver.cc:460] Webserver started at http://127.18.71.66:36349/ using document root <none> and password file <none>
I20260809 07:48:24.957830 20799 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:24.957924 20799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:24.958168 20799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:24.960271 20799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "d82d07afce67ef413e57d1507483bf7e"
server_key_iv: "eca7658335fdb40ba8d971f89be1b2c1"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.960918 20799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "d82d07afce67ef413e57d1507483bf7e"
server_key_iv: "eca7658335fdb40ba8d971f89be1b2c1"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.964833 20799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:48:24.967295 20815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:24.968616 20799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:24.968750 20799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6"
format_stamp: "Formatted at 2026-08-09 07:48:24 on dist-test-slave-10pc"
server_key: "d82d07afce67ef413e57d1507483bf7e"
server_key_iv: "eca7658335fdb40ba8d971f89be1b2c1"
server_key_version: "encryptionkey@0"
I20260809 07:48:24.968868 20799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:24.979561 20799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:24.982481 20799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:24.982692 20799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:24.984126 20799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:24.984208 20799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:24.984277 20799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:24.984329 20799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:24.994102 20799 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:39513
I20260809 07:48:24.994508 20934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:39513 every 8 connection(s)
Aug 09 07:48:24 dist-test-slave-10pc krb5kdc[20523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:24.995296 20799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:24.998867 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:24.985098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:58705 (local address 127.18.71.126:42059)
0809 07:48:24.985252 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:24.985256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:24.986329 (+ 1073us) server_negotiation.cc:407] Connection header received
0809 07:48:24.987512 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:24.987519 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:24.987593 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:24.987711 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:24.989540 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.990446 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.991966 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.992365 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.992536 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:24.995489 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:24.995492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:24.995495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:24.995521 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:24.997113 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.997579 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.997582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.997583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.997663 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:24.997948 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:24.997950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:24.997951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:24.998127 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:24.998213 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:24.998650 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 07:48:24.998750 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 07:48:24.999712 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:24.985645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42059 (local address 127.18.71.66:58705)
0809 07:48:24.986174 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:24.986218 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:24.987247 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:24.988042 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:24.988053 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:24.988609 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:24.989349 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:24.989371 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.990610 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.990614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:24.991807 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:24.991817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:24.992569 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:24.992574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:24.992795 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:24.993487 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:24.993503 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:24.995281 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:24.997228 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:24.997232 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:24.997244 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:24.997492 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:24.997743 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:24.997745 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:24.997746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:24.997871 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:24.998224 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:24.998229 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:24.998445 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 07:48:24.998595 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 07:48:24.998850 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":153,"threads_started":1}
I20260809 07:48:25.001037 20935 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42059
I20260809 07:48:25.001204 20935 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:25.001888 20935 heartbeater.cc:511] Master 127.18.71.126:42059 requested a full tablet report, sending...
I20260809 07:48:25.003077 20570 ts_manager.cc:194] Registered new tserver with Master: 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513)
I20260809 07:48:25.003389 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 20799
I20260809 07:48:25.003499 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:25.003784 18717 external_mini_cluster.cc:1428] Setting key f2072d85e44dc56b147dfb7a5ea99554
I20260809 07:48:25.003903 20570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:58705
I20260809 07:48:25.012897 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.004510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:55327 (local address 127.18.71.126:42059)
0809 07:48:25.004607 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.004614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.005145 (+ 531us) server_negotiation.cc:407] Connection header received
0809 07:48:25.005324 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.005328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.005386 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.005472 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:25.005983 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.006774 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.007820 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.008029 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.008109 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.009363 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:25.009366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:25.009368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:25.009390 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:25.011120 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.011632 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.011634 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.011636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.011714 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.012030 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.012032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.012033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.012232 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:25.012316 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.012685 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.012779 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 07:48:25.013053 20942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.004371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42059 (local address 127.18.71.66:55327)
0809 07:48:25.005062 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.005078 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.005211 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.005507 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.005510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.005736 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:25.005853 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.005861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.006905 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.006907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.007694 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.007706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.008144 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.008149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.008265 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.008855 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:25.008876 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:25.009219 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:25.011243 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.011246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.011249 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.011532 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.011813 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.011819 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.011824 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.011938 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.012343 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:25.012349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:25.012500 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.012639 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.012817 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":158,"threads_started":1}
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:25.035434 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42059
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:25.167198 20945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:25.167472 20945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:25.167572 20945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:25.171586 20945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:25.171679 20945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:25.171746 20945 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 07:48:25.171787 20945 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:25.171857 20945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:25.177255 20945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:42059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:25.178330 20945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:25.179374 20945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:25.189146 20951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:25.189194 20952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:25.189206 20954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:25.189378 20945 server_base.cc:1034] running on GCE node
I20260809 07:48:25.189865 20945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:25.190618 20945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:25.191843 20945 hybrid_clock.cc:648] HybridClock initialized: now 1786261705191821 us; error 33 us; skew 500 ppm
Aug 09 07:48:25 dist-test-slave-10pc krb5kdc[20523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261705, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:25.199146 20945 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:25.200098 20945 webserver.cc:460] Webserver started at http://127.18.71.67:40677/ using document root <none> and password file <none>
I20260809 07:48:25.200803 20945 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:25.200896 20945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:25.201141 20945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:25.203527 20945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "d6b55fde94664685a35cfaf15d8b8054"
format_stamp: "Formatted at 2026-08-09 07:48:25 on dist-test-slave-10pc"
server_key: "21347b9fc2d1896bd7834e7c10c8c7cc"
server_key_iv: "65a6f154b546d088c357baad2213ce54"
server_key_version: "encryptionkey@0"
I20260809 07:48:25.204118 20945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "d6b55fde94664685a35cfaf15d8b8054"
format_stamp: "Formatted at 2026-08-09 07:48:25 on dist-test-slave-10pc"
server_key: "21347b9fc2d1896bd7834e7c10c8c7cc"
server_key_iv: "65a6f154b546d088c357baad2213ce54"
server_key_version: "encryptionkey@0"
I20260809 07:48:25.208055 20945 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 07:48:25.210522 20961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:25.211890 20945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:25.212045 20945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "d6b55fde94664685a35cfaf15d8b8054"
format_stamp: "Formatted at 2026-08-09 07:48:25 on dist-test-slave-10pc"
server_key: "21347b9fc2d1896bd7834e7c10c8c7cc"
server_key_iv: "65a6f154b546d088c357baad2213ce54"
server_key_version: "encryptionkey@0"
I20260809 07:48:25.212177 20945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:25.224812 20945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:25.227656 20945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:25.227869 20945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:25.229494 20945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:25.229575 20945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:25.229650 20945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:25.229693 20945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:25 dist-test-slave-10pc krb5kdc[20523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261705, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:25.241276 20945 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:42025
I20260809 07:48:25.241353 21082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:42025 every 8 connection(s)
I20260809 07:48:25.242234 20945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:25.243283 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 20945
I20260809 07:48:25.243428 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:25.243760 18717 external_mini_cluster.cc:1428] Setting key 0b1e51b5e8fba341fda964563ae2ede6
I20260809 07:48:25.244418 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.230305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:50449 (local address 127.18.71.126:42059)
0809 07:48:25.230475 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.230483 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.231301 (+ 818us) server_negotiation.cc:407] Connection header received
0809 07:48:25.232347 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.232350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.232409 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.232499 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:25.234162 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.235041 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.236802 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.237040 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.237174 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.240269 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:25.240273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:25.240276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:25.240304 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:25.241995 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.242577 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.242583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.242585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.242681 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.243114 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.243116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.243117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.243350 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:25.243452 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.244104 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.244236 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:48:25.245420 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.230669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42059 (local address 127.18.71.67:50449)
0809 07:48:25.231116 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.231183 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.232061 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.232951 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.232964 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.233367 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:25.233992 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.234013 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.235168 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.235172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.236678 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.236689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.237235 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.237240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.237425 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.238040 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:25.238060 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:25.240088 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:25.242116 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.242122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.242137 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.242474 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.242813 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.242816 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.242818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.242976 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.243496 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:25.243506 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:25.243810 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.244045 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.244375 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":131,"threads_started":1}
I20260809 07:48:25.246587 21083 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42059
I20260809 07:48:25.246727 21083 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:25.247362 21083 heartbeater.cc:511] Master 127.18.71.126:42059 requested a full tablet report, sending...
I20260809 07:48:25.248512 20570 ts_manager.cc:194] Registered new tserver with Master: d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025)
I20260809 07:48:25.249284 20570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:50449
I20260809 07:48:25.255816 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.249023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42059 (local address 127.18.71.67:58015)
0809 07:48:25.249278 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.249290 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.249371 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.249697 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.249699 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.249924 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:25.250053 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.250060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.250782 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.250785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.251526 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.251536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.251924 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.251926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.252060 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.252640 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:25.252658 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:25.252978 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:25.254501 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.254504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.254506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.254701 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.254978 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.254988 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.254990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.255080 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.255411 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:25.255413 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:25.255490 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.255575 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.255684 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":117,"threads_started":1}
I20260809 07:48:25.255825 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.248962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:58015 (local address 127.18.71.126:42059)
0809 07:48:25.249058 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.249062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.249364 (+ 302us) server_negotiation.cc:407] Connection header received
0809 07:48:25.249539 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.249542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.249593 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.249679 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:25.250178 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.250649 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.251628 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.251829 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.251900 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.253088 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:25.253090 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:25.253092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:25.253111 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:25.254393 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.254787 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.254790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.254791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.254864 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.255174 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.255176 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.255177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.255330 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:25.255404 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.255630 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.255720 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 07:48:25.257792 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 07:48:25 dist-test-slave-10pc krb5kdc[20523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:48:25 dist-test-slave-10pc krb5kdc[20523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:25.285066 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.277051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37598 (local address 127.18.71.126:42059)
0809 07:48:25.277190 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.277194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.277208 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:48:25.277249 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.277251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.277291 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.277360 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:25.278068 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.278597 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.279394 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.279593 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.279665 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.282354 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:25.282357 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:25.282360 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:25.282381 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:25.283639 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.284017 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.284020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.284021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.284089 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.284317 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.284319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.284320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.284537 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:25.284640 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.284840 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.284931 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:48:25.287994 20570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:25.291015 20570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:25.306203 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.300255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.18.71.66:39513)
0809 07:48:25.300789 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.300801 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.301048 (+ 247us) server_negotiation.cc:407] Connection header received
0809 07:48:25.301257 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.301263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.301429 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.301605 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.302101 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.303140 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.304522 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.305379 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.305519 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.305699 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.305952 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.306050 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":120,"threads_started":1}
I20260809 07:48:25.306258 21103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.299745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39513 (local address 127.0.0.1:38398)
0809 07:48:25.300958 (+ 1213us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.300972 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.301114 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.301846 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.301849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.301866 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.301981 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.301988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.303249 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.303253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.304362 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.304372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.305502 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.305505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.305666 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.305786 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.305904 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.306052 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1155,"mutex_wait_us":98,"thread_start_us":85,"threads_started":1}
I20260809 07:48:25.306852 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.299576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57760 (local address 127.18.71.65:35625)
0809 07:48:25.300207 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.300214 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.300422 (+ 208us) server_negotiation.cc:407] Connection header received
0809 07:48:25.300692 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.300698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.300959 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.301110 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.301710 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.302778 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.304701 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.305464 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.305563 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.305681 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.306438 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.306690 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":107,"threads_started":1}
I20260809 07:48:25.307475 21106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.299431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:42025 (local address 127.0.0.1:43000)
0809 07:48:25.300212 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.300265 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.300394 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.301396 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.301399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.301415 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.301535 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.301541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.302984 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.302988 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.304578 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.304589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.307059 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.307064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.307157 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.307187 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.307280 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.307349 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":184,"thread_start_us":93,"threads_started":1}
I20260809 07:48:25.307549 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.300378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.18.71.67:42025)
0809 07:48:25.300887 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.300899 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.300922 (+ 23us) server_negotiation.cc:407] Connection header received
0809 07:48:25.301012 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.301017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.301228 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.301377 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.301657 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.302871 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.305748 (+ 2877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.306703 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.306804 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.306973 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.307313 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.307400 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":149,"threads_started":1}
I20260809 07:48:25.306258 21104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.299411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:35625 (local address 127.0.0.1:57760)
0809 07:48:25.300212 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.300265 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.300511 (+ 246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.301291 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.301297 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.301318 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.301506 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.301514 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.302885 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.302891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.304579 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.304589 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.305571 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.305574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.305704 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.305786 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.305904 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.306052 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":192,"thread_start_us":73,"threads_started":1}
I20260809 07:48:25.309525 20852 tablet_service.cc:1467] Processing CreateTablet for tablet d6532f666215452db77284e0e143b2ad (DEFAULT_TABLE table=test-table [id=a528d43ce7614ae6ae49d2d9cb6599e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:25.310223 21012 tablet_service.cc:1467] Processing CreateTablet for tablet d6532f666215452db77284e0e143b2ad (DEFAULT_TABLE table=test-table [id=a528d43ce7614ae6ae49d2d9cb6599e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:25.310686 20852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6532f666215452db77284e0e143b2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.311252 21012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6532f666215452db77284e0e143b2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.311003 20719 tablet_service.cc:1467] Processing CreateTablet for tablet d6532f666215452db77284e0e143b2ad (DEFAULT_TABLE table=test-table [id=a528d43ce7614ae6ae49d2d9cb6599e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:25.311817 20719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6532f666215452db77284e0e143b2ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.317649 21109 tablet_bootstrap.cc:492] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Bootstrap starting.
I20260809 07:48:25.319433 21110 tablet_bootstrap.cc:492] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Bootstrap starting.
I20260809 07:48:25.319881 21109 tablet_bootstrap.cc:654] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.320636 21109 log.cc:824] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:25.321354 21111 tablet_bootstrap.cc:492] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Bootstrap starting.
I20260809 07:48:25.321398 21110 tablet_bootstrap.cc:654] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.322105 21110 log.cc:824] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:25.323030 21111 tablet_bootstrap.cc:654] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.323688 21111 log.cc:824] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:25.323974 21110 tablet_bootstrap.cc:492] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: No bootstrap required, opened a new log
I20260809 07:48:25.324179 21110 ts_tablet_manager.cc:1397] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:48:25.325193 21109 tablet_bootstrap.cc:492] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: No bootstrap required, opened a new log
I20260809 07:48:25.325392 21109 ts_tablet_manager.cc:1397] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20260809 07:48:25.325654 21111 tablet_bootstrap.cc:492] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: No bootstrap required, opened a new log
I20260809 07:48:25.325827 21111 ts_tablet_manager.cc:1397] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:48:25.328294 21110 raft_consensus.cc:348] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.328614 21110 raft_consensus.cc:374] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.328712 21110 raft_consensus.cc:729] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b55fde94664685a35cfaf15d8b8054, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.329180 21110 consensus_queue.cc:260] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.329550 21109 raft_consensus.cc:348] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.329821 21109 raft_consensus.cc:374] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.329891 21109 raft_consensus.cc:729] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37da323536664cf2af59ba85eb42d4eb, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.329792 21111 raft_consensus.cc:348] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.330046 21111 raft_consensus.cc:374] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.330163 21111 raft_consensus.cc:729] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e4f0d1a8da4a1a9b26b502d82f02a6, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.330221 21110 ts_tablet_manager.cc:1428] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:48:25.330574 21083 heartbeater.cc:503] Master 127.18.71.126:42059 was elected leader, sending a full tablet report...
I20260809 07:48:25.330617 21111 consensus_queue.cc:260] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.331218 20935 heartbeater.cc:503] Master 127.18.71.126:42059 was elected leader, sending a full tablet report...
I20260809 07:48:25.330319 21109 consensus_queue.cc:260] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.332356 21109 ts_tablet_manager.cc:1428] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 07:48:25.332536 21111 ts_tablet_manager.cc:1428] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 07:48:25.332731 20788 heartbeater.cc:503] Master 127.18.71.126:42059 was elected leader, sending a full tablet report...
I20260809 07:48:25.393612 21115 raft_consensus.cc:482] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:25.393821 21115 raft_consensus.cc:504] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.394949 21115 leader_election.cc:288] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625), 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513)
I20260809 07:48:25.399318 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.395135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:35625 (local address 127.18.71.67:49055)
0809 07:48:25.395255 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.395273 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.395398 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.395811 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.395814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.395832 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.395956 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.395962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.397063 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.397066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.398044 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.398053 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.398862 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.398865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.398973 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.398999 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.399079 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.399163 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":42}
I20260809 07:48:25.399338 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.395457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:49055 (local address 127.18.71.65:35625)
0809 07:48:25.395587 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.395592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.395607 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:25.395664 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.395667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.395714 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.395788 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.396096 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.396947 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.398182 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.398751 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.398842 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.398882 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.399107 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.399203 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:48:25.399608 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.395222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39513 (local address 127.18.71.67:47501)
0809 07:48:25.395647 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.395664 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.395790 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.396096 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.396099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.396117 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.396234 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.396239 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.397526 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.397528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.398441 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.398451 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.399187 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.399191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.399287 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.399309 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.399391 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.399477 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":1,"thread_start_us":135,"threads_started":1}
I20260809 07:48:25.399744 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.395433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:47501 (local address 127.18.71.66:39513)
0809 07:48:25.395589 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.395594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.395749 (+ 155us) server_negotiation.cc:407] Connection header received
0809 07:48:25.395936 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.395940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.395993 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.396072 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.396760 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.397409 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.398576 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.399076 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.399187 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.399232 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.399558 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.399632 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 07:48:25.400322 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6532f666215452db77284e0e143b2ad" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" is_pre_election: true
I20260809 07:48:25.400390 20739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6532f666215452db77284e0e143b2ad" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37da323536664cf2af59ba85eb42d4eb" is_pre_election: true
I20260809 07:48:25.400689 20886 raft_consensus.cc:2454] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 0.
I20260809 07:48:25.400743 20739 raft_consensus.cc:2454] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 0.
I20260809 07:48:25.401196 20964 leader_election.cc:302] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98e4f0d1a8da4a1a9b26b502d82f02a6, d6b55fde94664685a35cfaf15d8b8054; no voters:
I20260809 07:48:25.401448 21115 raft_consensus.cc:2781] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:25.401535 21115 raft_consensus.cc:482] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:25.401579 21115 raft_consensus.cc:3037] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.402808 21115 raft_consensus.cc:504] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.403142 21115 leader_election.cc:288] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 election: Requested vote from peers 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625), 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513)
I20260809 07:48:25.403465 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6532f666215452db77284e0e143b2ad" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6"
I20260809 07:48:25.403565 20886 raft_consensus.cc:3037] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.404620 20739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6532f666215452db77284e0e143b2ad" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37da323536664cf2af59ba85eb42d4eb"
I20260809 07:48:25.404764 20739 raft_consensus.cc:3037] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.404785 20886 raft_consensus.cc:2454] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 1.
I20260809 07:48:25.405135 20964 leader_election.cc:302] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98e4f0d1a8da4a1a9b26b502d82f02a6, d6b55fde94664685a35cfaf15d8b8054; no voters:
I20260809 07:48:25.405293 21115 raft_consensus.cc:2781] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:25.405514 21115 raft_consensus.cc:686] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 1 LEADER]: Becoming Leader. State: Replica: d6b55fde94664685a35cfaf15d8b8054, State: Running, Role: LEADER
I20260809 07:48:25.405840 21115 consensus_queue.cc:237] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } }
I20260809 07:48:25.406126 20739 raft_consensus.cc:2454] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 1.
I20260809 07:48:25.409235 20570 catalog_manager.cc:5495] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "d6b55fde94664685a35cfaf15d8b8054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } health_report { overall_health: UNKNOWN } } }
Aug 09 07:48:25 dist-test-slave-10pc krb5kdc[20523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261704, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:25.450747 21126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.442282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42059 (local address 127.0.0.1:37600)
0809 07:48:25.442555 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.442569 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.442675 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.442900 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.442903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.443275 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:25.443361 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.443367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.444074 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.444076 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.444632 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.444639 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.445120 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.445123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.445186 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.445721 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:25.445732 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:25.447530 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:25.449220 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.449229 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.449234 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.449485 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.449761 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:25.449764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:25.449765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:25.449856 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:25.450217 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:25.450223 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:25.450327 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.450430 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.450565 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":112,"threads_started":1}
I20260809 07:48:25.450749 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.442319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37600 (local address 127.18.71.126:42059)
0809 07:48:25.442456 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.442461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.442641 (+ 180us) server_negotiation.cc:407] Connection header received
0809 07:48:25.442752 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.442754 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.442792 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.442862 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:25.443470 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.443975 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.444734 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.444940 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.445100 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.447700 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:25.447703 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:25.447705 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:25.447726 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:25.449116 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.449578 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.449581 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.449582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.449650 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:25.449944 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:25.449946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:25.449947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:25.450104 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:25.450177 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.450484 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.450618 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":54}
I20260809 07:48:25.454540 20570 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:37600:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 07:48:25.455271 20570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:25.459966 20719 tablet_service.cc:1467] Processing CreateTablet for tablet 57c82d3c2ffc42168fcabd0bc72b8e9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e7a275125fb4e8fac06c4cf98e19ff9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:25.460038 21012 tablet_service.cc:1467] Processing CreateTablet for tablet 57c82d3c2ffc42168fcabd0bc72b8e9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e7a275125fb4e8fac06c4cf98e19ff9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:25.460320 20852 tablet_service.cc:1467] Processing CreateTablet for tablet 57c82d3c2ffc42168fcabd0bc72b8e9a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1e7a275125fb4e8fac06c4cf98e19ff9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:25.460801 20719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c82d3c2ffc42168fcabd0bc72b8e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.460803 21012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c82d3c2ffc42168fcabd0bc72b8e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.460912 20852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57c82d3c2ffc42168fcabd0bc72b8e9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.463604 21109 tablet_bootstrap.cc:492] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: Bootstrap starting.
I20260809 07:48:25.463670 21110 tablet_bootstrap.cc:492] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: Bootstrap starting.
I20260809 07:48:25.464637 21109 tablet_bootstrap.cc:654] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.464623 21110 tablet_bootstrap.cc:654] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.465440 21111 tablet_bootstrap.cc:492] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: Bootstrap starting.
I20260809 07:48:25.466368 21111 tablet_bootstrap.cc:654] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.466497 21109 tablet_bootstrap.cc:492] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: No bootstrap required, opened a new log
I20260809 07:48:25.466488 21110 tablet_bootstrap.cc:492] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: No bootstrap required, opened a new log
I20260809 07:48:25.466571 21109 ts_tablet_manager.cc:1397] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:48:25.466571 21110 ts_tablet_manager.cc:1397] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:48:25.467024 21109 raft_consensus.cc:348] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.467025 21110 raft_consensus.cc:348] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.467135 21109 raft_consensus.cc:374] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.467135 21110 raft_consensus.cc:374] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.467170 21109 raft_consensus.cc:729] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37da323536664cf2af59ba85eb42d4eb, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.467170 21110 raft_consensus.cc:729] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b55fde94664685a35cfaf15d8b8054, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.467300 21109 consensus_queue.cc:260] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.467300 21110 consensus_queue.cc:260] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.467514 21111 tablet_bootstrap.cc:492] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: No bootstrap required, opened a new log
I20260809 07:48:25.467562 21110 ts_tablet_manager.cc:1428] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:25.467569 21109 ts_tablet_manager.cc:1428] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:25.467594 21111 ts_tablet_manager.cc:1397] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:25.468016 21111 raft_consensus.cc:348] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.468137 21111 raft_consensus.cc:374] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.467561 21117 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.468182 21111 raft_consensus.cc:729] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e4f0d1a8da4a1a9b26b502d82f02a6, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.468217 21117 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:25.468300 21111 consensus_queue.cc:260] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.468413 21115 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.468567 21115 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:25.468607 21111 ts_tablet_manager.cc:1428] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:25.468767 21116 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.468889 21116 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 07:48:25.493283 21085 tablet.cc:2369] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:48:25.496469 20937 tablet.cc:2369] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:25.510059 21116 raft_consensus.cc:482] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:25.510187 21116 raft_consensus.cc:504] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.511204 21116 leader_election.cc:288] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625), d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025)
I20260809 07:48:25.515166 21130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.511421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:35625 (local address 127.18.71.66:60985)
0809 07:48:25.511701 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.511716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.511814 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.512194 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.512200 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.512220 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.512352 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.512358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.513133 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.513136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.514020 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.514030 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.514715 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.514718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.514834 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.514861 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.514939 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.515004 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":103,"threads_started":1}
I20260809 07:48:25.515178 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.511800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:60985 (local address 127.18.71.65:35625)
0809 07:48:25.511965 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.511970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.511985 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:25.512041 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.512043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.512096 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.512154 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.512485 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.513023 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.514144 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.514595 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.514691 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.514731 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.514966 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.515058 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 07:48:25.515280 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.511548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:51599 (local address 127.18.71.67:42025)
0809 07:48:25.511682 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.511686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.511960 (+ 274us) server_negotiation.cc:407] Connection header received
0809 07:48:25.512073 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.512075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.512117 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.512212 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.512587 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.513120 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.514267 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.514719 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.514843 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.514888 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.515076 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.515153 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 07:48:25.515331 21131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.511421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:42025 (local address 127.18.71.66:51599)
0809 07:48:25.511827 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.511862 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.511977 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.512240 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.512244 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.512315 (+ 71us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.512436 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.512444 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.513230 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.513233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.514132 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.514142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.514839 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.514842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.514942 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.514967 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.515042 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.515104 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":74,"spinlock_wait_cycles":19712,"thread_start_us":81,"threads_started":1}
I20260809 07:48:25.515761 20739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c82d3c2ffc42168fcabd0bc72b8e9a" candidate_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37da323536664cf2af59ba85eb42d4eb" is_pre_election: true
I20260809 07:48:25.515940 20739 raft_consensus.cc:2454] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98e4f0d1a8da4a1a9b26b502d82f02a6 in term 0.
I20260809 07:48:25.515983 21033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c82d3c2ffc42168fcabd0bc72b8e9a" candidate_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b55fde94664685a35cfaf15d8b8054" is_pre_election: true
I20260809 07:48:25.516234 21033 raft_consensus.cc:2454] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98e4f0d1a8da4a1a9b26b502d82f02a6 in term 0.
I20260809 07:48:25.516366 20816 leader_election.cc:302] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37da323536664cf2af59ba85eb42d4eb, 98e4f0d1a8da4a1a9b26b502d82f02a6; no voters:
I20260809 07:48:25.516706 21116 raft_consensus.cc:2781] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:25.516777 21116 raft_consensus.cc:482] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:25.516825 21116 raft_consensus.cc:3037] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.517807 21116 raft_consensus.cc:504] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.518141 21116 leader_election.cc:288] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [CANDIDATE]: Term 1 election: Requested vote from peers 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625), d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025)
I20260809 07:48:25.518462 20739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c82d3c2ffc42168fcabd0bc72b8e9a" candidate_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37da323536664cf2af59ba85eb42d4eb"
I20260809 07:48:25.518538 21033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57c82d3c2ffc42168fcabd0bc72b8e9a" candidate_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b55fde94664685a35cfaf15d8b8054"
I20260809 07:48:25.518610 20739 raft_consensus.cc:3037] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.518671 21033 raft_consensus.cc:3037] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.519517 20739 raft_consensus.cc:2454] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98e4f0d1a8da4a1a9b26b502d82f02a6 in term 1.
I20260809 07:48:25.519678 21033 raft_consensus.cc:2454] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98e4f0d1a8da4a1a9b26b502d82f02a6 in term 1.
I20260809 07:48:25.519791 20816 leader_election.cc:302] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37da323536664cf2af59ba85eb42d4eb, 98e4f0d1a8da4a1a9b26b502d82f02a6; no voters:
I20260809 07:48:25.519984 21116 raft_consensus.cc:2781] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:25.520196 21116 raft_consensus.cc:686] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 LEADER]: Becoming Leader. State: Replica: 98e4f0d1a8da4a1a9b26b502d82f02a6, State: Running, Role: LEADER
I20260809 07:48:25.520579 21116 consensus_queue.cc:237] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.523160 21132 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: TxnStatusTablet state changed. Reason: New leader 98e4f0d1a8da4a1a9b26b502d82f02a6. Latest consensus state: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.523301 21132 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:25.523607 21134 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 07:48:25.523870 21134 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 07:48:25.524298 20568 catalog_manager.cc:5495] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } health_report { overall_health: UNKNOWN } } }
W20260809 07:48:25.528597 20789 tablet.cc:2369] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:25.546126 21126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.542422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39513 (local address 127.0.0.1:38400)
0809 07:48:25.542572 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.542591 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.542712 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.543051 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.543054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.543061 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:25.543167 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.543172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.543877 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.543881 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.544551 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.544560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.545021 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.545024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.545129 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.545171 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.545821 (+ 650us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.545838 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.545896 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.545997 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 07:48:25.546126 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.542558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.18.71.66:39513)
0809 07:48:25.542688 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.542692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.542707 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:25.542866 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.542869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.542929 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.543024 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:25.543273 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.543766 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.544669 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.544894 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.545022 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.545319 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.545719 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.545870 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.545915 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.545970 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:48:25.551530 21033 raft_consensus.cc:1261] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Refusing update from remote peer 98e4f0d1a8da4a1a9b26b502d82f02a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:25.551630 20739 raft_consensus.cc:1261] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Refusing update from remote peer 98e4f0d1a8da4a1a9b26b502d82f02a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:25.552166 21116 consensus_queue.cc:1035] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:25.552364 21132 consensus_queue.cc:1035] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:25.555536 21115 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: TxnStatusTablet state changed. Reason: New leader 98e4f0d1a8da4a1a9b26b502d82f02a6. Latest consensus state: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.555666 21115 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:25.555835 21117 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: TxnStatusTablet state changed. Reason: New leader 98e4f0d1a8da4a1a9b26b502d82f02a6. Latest consensus state: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.555943 21117 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:25.558990 21132 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.559113 21132 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:25.559252 21132 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.559343 21132 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 98e4f0d1a8da4a1a9b26b502d82f02a6: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:25.559897 21137 mvcc.cc:205] Tried to move back new op lower bound from 7316527945932718080 to 7316527945815375872. Current Snapshot: MvccSnapshot[applied={T|T < 7316527945932718080}]
I20260809 07:48:25.559949 21115 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.560045 21115 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P d6b55fde94664685a35cfaf15d8b8054: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:25.560199 21117 tablet_replica.cc:439] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } } }
I20260809 07:48:25.560295 21117 tablet_replica.cc:442] T 57c82d3c2ffc42168fcabd0bc72b8e9a P 37da323536664cf2af59ba85eb42d4eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:25.561491 21138 mvcc.cc:205] Tried to move back new op lower bound from 7316527945932718080 to 7316527945815375872. Current Snapshot: MvccSnapshot[applied={T|T < 7316527945932718080}]
I20260809 07:48:25.562090 21139 mvcc.cc:205] Tried to move back new op lower bound from 7316527945932718080 to 7316527945815375872. Current Snapshot: MvccSnapshot[applied={T|T < 7316527945932718080}]
I20260809 07:48:25.571969 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.568032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.18.71.67:42025)
0809 07:48:25.568149 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.568155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.568171 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:48:25.568338 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.568342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.568401 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.568555 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:25.568773 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.569556 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.570388 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.570741 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.570844 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.571067 (+ 223us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.571572 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.571685 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.571740 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.571796 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 07:48:25.583462 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.579694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39513 (local address 127.18.71.67:45971)
0809 07:48:25.579965 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.579978 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.580051 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.580305 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.580307 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.580312 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:25.580425 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.580429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.581449 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.581452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.582198 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.582207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.582763 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.582765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.582835 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.582879 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.583233 (+ 354us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.583239 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.583293 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.583355 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 07:48:25.583516 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.579838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:45971 (local address 127.18.71.66:39513)
0809 07:48:25.579974 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.579979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.580017 (+ 38us) server_negotiation.cc:407] Connection header received
0809 07:48:25.580144 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.580148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.580201 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.580292 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:25.580555 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.581331 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.582309 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.582657 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.582762 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.583064 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.583146 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.583220 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.583330 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.583386 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:48:25.590314 20739 raft_consensus.cc:1261] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Refusing update from remote peer d6b55fde94664685a35cfaf15d8b8054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:25.590411 20886 raft_consensus.cc:1261] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Refusing update from remote peer d6b55fde94664685a35cfaf15d8b8054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:25.590826 21119 consensus_queue.cc:1035] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:25.591014 21115 consensus_queue.cc:1035] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:25.615773 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.611754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:39221 (local address 127.18.71.67:42025)
0809 07:48:25.611866 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.611871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.612115 (+ 244us) server_negotiation.cc:407] Connection header received
0809 07:48:25.612261 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.612264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.612314 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.612391 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:25.612718 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.613499 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.614593 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.614957 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.615067 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.615358 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.615437 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.615509 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.615587 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.615639 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 07:48:25.615828 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.611701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:42025 (local address 127.18.71.66:39221)
0809 07:48:25.612053 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.612067 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.612151 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.612429 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.612432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.612442 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:25.612613 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.612617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.613601 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.613604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.614482 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.614492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.615097 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.615100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.615189 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.615232 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.615522 (+ 290us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.615527 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.615575 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.615672 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":117,"threads_started":1}
I20260809 07:48:25.643962 20568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:25.644172 20568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:25.647210 20568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Sending DeleteTablet for 3 replicas of tablet d6532f666215452db77284e0e143b2ad
I20260809 07:48:25.647986 20852 tablet_service.cc:1514] Processing DeleteTablet for tablet d6532f666215452db77284e0e143b2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:25 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:38398
I20260809 07:48:25.648171 20719 tablet_service.cc:1514] Processing DeleteTablet for tablet d6532f666215452db77284e0e143b2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:25 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:57760
I20260809 07:48:25.649000 21012 tablet_service.cc:1514] Processing DeleteTablet for tablet d6532f666215452db77284e0e143b2ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:25 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:43000
I20260809 07:48:25.649068 21156 tablet_replica.cc:330] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: stopping tablet replica
I20260809 07:48:25.649461 21156 raft_consensus.cc:2229] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:25.649695 21156 raft_consensus.cc:2258] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:25.650019 20568 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:25.650434 20568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:25.651458 21158 tablet_replica.cc:330] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: stopping tablet replica
I20260809 07:48:25.651734 21158 raft_consensus.cc:2229] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:25.652104 21158 raft_consensus.cc:2258] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:25.654361 21156 ts_tablet_manager.cc:1905] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:25.654557 21158 ts_tablet_manager.cc:1905] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:25.656416 20719 tablet_service.cc:1467] Processing CreateTablet for tablet a1760e1f6f544ef283b50d65b3f477f1 (DEFAULT_TABLE table=test-table [id=06c80ac20cbc4306bcfcd05a5b5a5854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:25.656749 20719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1760e1f6f544ef283b50d65b3f477f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.657789 21156 ts_tablet_manager.cc:1918] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:25.657862 21158 ts_tablet_manager.cc:1918] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:25.657972 21158 log.cc:1196] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/d6532f666215452db77284e0e143b2ad
I20260809 07:48:25.658004 21156 log.cc:1196] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/d6532f666215452db77284e0e143b2ad
I20260809 07:48:25.658349 21158 ts_tablet_manager.cc:1939] T d6532f666215452db77284e0e143b2ad P d6b55fde94664685a35cfaf15d8b8054: Deleting consensus metadata
I20260809 07:48:25.658550 21156 ts_tablet_manager.cc:1939] T d6532f666215452db77284e0e143b2ad P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting consensus metadata
I20260809 07:48:25.659403 21109 tablet_bootstrap.cc:492] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: Bootstrap starting.
I20260809 07:48:25.659655 20852 tablet_service.cc:1467] Processing CreateTablet for tablet a1760e1f6f544ef283b50d65b3f477f1 (DEFAULT_TABLE table=test-table [id=06c80ac20cbc4306bcfcd05a5b5a5854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:25.659741 20555 catalog_manager.cc:4841] TS 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513): tablet d6532f666215452db77284e0e143b2ad (table test-table [id=a528d43ce7614ae6ae49d2d9cb6599e7]) successfully deleted
I20260809 07:48:25.659919 20852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1760e1f6f544ef283b50d65b3f477f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.660399 21109 tablet_bootstrap.cc:654] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.660637 21157 tablet_replica.cc:330] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: stopping tablet replica
I20260809 07:48:25.660945 21157 raft_consensus.cc:2229] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:25.661175 21157 raft_consensus.cc:2258] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:25.661779 21109 tablet_bootstrap.cc:492] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: No bootstrap required, opened a new log
I20260809 07:48:25.661876 21109 ts_tablet_manager.cc:1397] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:25.662192 21111 tablet_bootstrap.cc:492] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Bootstrap starting.
I20260809 07:48:25.662348 21012 tablet_service.cc:1467] Processing CreateTablet for tablet a1760e1f6f544ef283b50d65b3f477f1 (DEFAULT_TABLE table=test-table [id=06c80ac20cbc4306bcfcd05a5b5a5854]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:25.662339 21109 raft_consensus.cc:348] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.662449 21109 raft_consensus.cc:374] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.662480 21109 raft_consensus.cc:729] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37da323536664cf2af59ba85eb42d4eb, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.662581 21012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1760e1f6f544ef283b50d65b3f477f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:25.662613 21109 consensus_queue.cc:260] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.662904 21109 ts_tablet_manager.cc:1428] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:25.663508 21111 tablet_bootstrap.cc:654] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.664876 21111 tablet_bootstrap.cc:492] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: No bootstrap required, opened a new log
I20260809 07:48:25.664880 21110 tablet_bootstrap.cc:492] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: Bootstrap starting.
I20260809 07:48:25.664984 21111 ts_tablet_manager.cc:1397] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:25.665454 21111 raft_consensus.cc:348] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.665576 21111 raft_consensus.cc:374] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.665614 21111 raft_consensus.cc:729] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e4f0d1a8da4a1a9b26b502d82f02a6, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.665735 21111 consensus_queue.cc:260] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.665831 21110 tablet_bootstrap.cc:654] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:25.666002 21111 ts_tablet_manager.cc:1428] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:25.666714 20554 catalog_manager.cc:4841] TS d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025): tablet d6532f666215452db77284e0e143b2ad (table test-table [id=a528d43ce7614ae6ae49d2d9cb6599e7]) successfully deleted
I20260809 07:48:25.666976 21110 tablet_bootstrap.cc:492] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: No bootstrap required, opened a new log
I20260809 07:48:25.667062 21110 ts_tablet_manager.cc:1397] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 07:48:25.667484 21110 raft_consensus.cc:348] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.667586 21110 raft_consensus.cc:374] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:25.667618 21110 raft_consensus.cc:729] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b55fde94664685a35cfaf15d8b8054, State: Initialized, Role: FOLLOWER
I20260809 07:48:25.667760 21110 consensus_queue.cc:260] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.668025 21110 ts_tablet_manager.cc:1428] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:25.669772 21157 ts_tablet_manager.cc:1905] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:25.672403 21157 ts_tablet_manager.cc:1918] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:25.672538 21157 log.cc:1196] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/d6532f666215452db77284e0e143b2ad
I20260809 07:48:25.672839 21157 ts_tablet_manager.cc:1939] T d6532f666215452db77284e0e143b2ad P 37da323536664cf2af59ba85eb42d4eb: Deleting consensus metadata
I20260809 07:48:25.673976 20552 catalog_manager.cc:4841] TS 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625): tablet d6532f666215452db77284e0e143b2ad (table test-table [id=a528d43ce7614ae6ae49d2d9cb6599e7]) successfully deleted
I20260809 07:48:25.755550 21117 raft_consensus.cc:482] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:25.755694 21117 raft_consensus.cc:504] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.756829 21117 leader_election.cc:288] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513), d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025)
I20260809 07:48:25.762071 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.757384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:41411 (local address 127.18.71.66:39513)
0809 07:48:25.757514 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.757519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.757535 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:48:25.757620 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.757623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.757677 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.757748 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.758096 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.758920 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.760874 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.761499 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.761633 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.761688 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.761866 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.761951 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:48:25.762102 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.757053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39513 (local address 127.18.71.65:41411)
0809 07:48:25.757391 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.757407 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.757514 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.757805 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.757808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.757822 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.757980 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.757990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.759072 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.759078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.760735 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.760743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.761644 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.761647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.761728 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.761749 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.761842 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.761919 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":94,"thread_start_us":97,"threads_started":1}
I20260809 07:48:25.762914 21108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.757296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:41215 (local address 127.18.71.67:42025)
0809 07:48:25.757412 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.757417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.757609 (+ 192us) server_negotiation.cc:407] Connection header received
0809 07:48:25.757789 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.757792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.757840 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.757916 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:25.758275 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.759012 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.761750 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.762356 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.762476 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.762521 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.762732 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.762809 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 07:48:25.762985 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1760e1f6f544ef283b50d65b3f477f1" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" is_pre_election: true
I20260809 07:48:25.763172 20886 raft_consensus.cc:2454] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 0.
I20260809 07:48:25.763259 21162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.757080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:42025 (local address 127.18.71.65:41215)
0809 07:48:25.757531 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.757545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.757669 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.757953 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.757956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.757979 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:25.758110 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.758116 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.759138 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.759141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.760731 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.760743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.762481 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.762484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.762594 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.762613 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.762706 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.762772 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":187,"thread_start_us":75,"threads_started":1}
I20260809 07:48:25.763587 20666 leader_election.cc:302] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37da323536664cf2af59ba85eb42d4eb, 98e4f0d1a8da4a1a9b26b502d82f02a6; no voters:
I20260809 07:48:25.763793 21033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1760e1f6f544ef283b50d65b3f477f1" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b55fde94664685a35cfaf15d8b8054" is_pre_election: true
I20260809 07:48:25.763861 21117 raft_consensus.cc:2781] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:25.763918 21117 raft_consensus.cc:482] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:25.763959 21117 raft_consensus.cc:3037] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.763942 21033 raft_consensus.cc:2454] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 0.
I20260809 07:48:25.765146 21117 raft_consensus.cc:504] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.765465 21117 leader_election.cc:288] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 election: Requested vote from peers 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513), d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025)
I20260809 07:48:25.765779 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1760e1f6f544ef283b50d65b3f477f1" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6"
I20260809 07:48:25.765821 21033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1760e1f6f544ef283b50d65b3f477f1" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b55fde94664685a35cfaf15d8b8054"
I20260809 07:48:25.765913 20886 raft_consensus.cc:3037] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.765954 21033 raft_consensus.cc:3037] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:25.766848 21033 raft_consensus.cc:2454] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 1.
I20260809 07:48:25.767095 20886 raft_consensus.cc:2454] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 1.
I20260809 07:48:25.767139 20665 leader_election.cc:302] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37da323536664cf2af59ba85eb42d4eb, d6b55fde94664685a35cfaf15d8b8054; no voters:
I20260809 07:48:25.767304 21117 raft_consensus.cc:2781] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:25.767567 21117 raft_consensus.cc:686] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 1 LEADER]: Becoming Leader. State: Replica: 37da323536664cf2af59ba85eb42d4eb, State: Running, Role: LEADER
I20260809 07:48:25.767820 21117 consensus_queue.cc:237] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:25.770576 20567 catalog_manager.cc:5495] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 37da323536664cf2af59ba85eb42d4eb (127.18.71.65). New cstate: current_term: 1 leader_uuid: "37da323536664cf2af59ba85eb42d4eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } health_report { overall_health: UNKNOWN } } }
I20260809 07:48:25.801992 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.798373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57774 (local address 127.18.71.65:35625)
0809 07:48:25.798488 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.798493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.798506 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:48:25.798579 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.798581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.798641 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.798749 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:25.799012 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.799505 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.800290 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.800665 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.800776 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.801048 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.801572 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.801691 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.801787 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.801844 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:48:25.812256 21107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.809017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:38445 (local address 127.18.71.66:39513)
0809 07:48:25.809133 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.809138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.809158 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:48:25.809272 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.809274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.809315 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.809373 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:25.809684 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.810158 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.811311 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.811518 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.811593 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.811882 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.811952 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.812010 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.812113 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.812156 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:48:25.812363 21168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.808741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39513 (local address 127.18.71.65:38445)
0809 07:48:25.809062 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.809078 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.809178 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.809413 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.809418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.809428 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:25.809551 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.809560 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.810269 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.810272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.811203 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.811213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.811625 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.811629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.811712 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.811762 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.812030 (+ 268us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.812036 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.812096 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.812178 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260809 07:48:25.819308 21033 raft_consensus.cc:1261] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Refusing update from remote peer 37da323536664cf2af59ba85eb42d4eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:25.819569 20886 raft_consensus.cc:1261] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Refusing update from remote peer 37da323536664cf2af59ba85eb42d4eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:25.819905 21164 consensus_queue.cc:1035] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:25.820345 21117 consensus_queue.cc:1035] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:25.839458 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.835893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:35625 (local address 127.18.71.66:42681)
0809 07:48:25.836064 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:25.836081 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:25.836171 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:25.836667 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:25.836670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:25.836679 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:25.836789 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.836795 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.837716 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.837719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.838304 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:25.838312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.838776 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.838779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:25.838866 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.838900 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.839188 (+ 288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.839198 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 07:48:25.839264 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 07:48:25.839350 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 07:48:25.839448 21105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:25.835934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:42681 (local address 127.18.71.65:35625)
0809 07:48:25.836431 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:25.836435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:25.836447 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:48:25.836512 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:25.836514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:25.836557 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:25.836624 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:25.836911 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.837597 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.838425 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:25.838664 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:25.838772 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:25.839027 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:25.839103 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:25.839181 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:25.839272 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 07:48:25.839317 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422}
W20260809 07:48:25.840196 21139 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 07:48:27.398662 20567 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:27.398809 20567 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:27.401134 20567 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Sending DeleteTablet for 3 replicas of tablet a1760e1f6f544ef283b50d65b3f477f1
I20260809 07:48:27.401629 20719 tablet_service.cc:1514] Processing DeleteTablet for tablet a1760e1f6f544ef283b50d65b3f477f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:27 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:57760
I20260809 07:48:27.401690 20852 tablet_service.cc:1514] Processing DeleteTablet for tablet a1760e1f6f544ef283b50d65b3f477f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:27 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:38398
I20260809 07:48:27.401983 21012 tablet_service.cc:1514] Processing DeleteTablet for tablet a1760e1f6f544ef283b50d65b3f477f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:27 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:43000
I20260809 07:48:27.402366 21203 tablet_replica.cc:330] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: stopping tablet replica
I20260809 07:48:27.402503 21202 tablet_replica.cc:330] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: stopping tablet replica
I20260809 07:48:27.402566 21203 raft_consensus.cc:2229] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:27.402647 21202 raft_consensus.cc:2229] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:27.402686 21203 raft_consensus.cc:2258] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:27.402768 21202 raft_consensus.cc:2258] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:27.402145 21201 tablet_replica.cc:330] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: stopping tablet replica
I20260809 07:48:27.403092 21201 raft_consensus.cc:2229] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:27.403370 21203 ts_tablet_manager.cc:1905] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:27.403438 21202 ts_tablet_manager.cc:1905] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:27.403587 21201 raft_consensus.cc:2258] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:27.404178 21201 ts_tablet_manager.cc:1905] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:27.406303 21203 ts_tablet_manager.cc:1918] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 07:48:27.406387 21203 log.cc:1196] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/a1760e1f6f544ef283b50d65b3f477f1
I20260809 07:48:27.406675 21203 ts_tablet_manager.cc:1939] T a1760e1f6f544ef283b50d65b3f477f1 P d6b55fde94664685a35cfaf15d8b8054: Deleting consensus metadata
I20260809 07:48:27.406685 21202 ts_tablet_manager.cc:1918] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 07:48:27.406764 21202 log.cc:1196] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/a1760e1f6f544ef283b50d65b3f477f1
I20260809 07:48:27.407063 21202 ts_tablet_manager.cc:1939] T a1760e1f6f544ef283b50d65b3f477f1 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting consensus metadata
I20260809 07:48:27.407363 20554 catalog_manager.cc:4841] TS d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025): tablet a1760e1f6f544ef283b50d65b3f477f1 (table test-table [id=06c80ac20cbc4306bcfcd05a5b5a5854]) successfully deleted
I20260809 07:48:27.407519 21201 ts_tablet_manager.cc:1918] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 07:48:27.407608 21201 log.cc:1196] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/a1760e1f6f544ef283b50d65b3f477f1
I20260809 07:48:27.407756 20555 catalog_manager.cc:4841] TS 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513): tablet a1760e1f6f544ef283b50d65b3f477f1 (table test-table [id=06c80ac20cbc4306bcfcd05a5b5a5854]) successfully deleted
I20260809 07:48:27.407912 21201 ts_tablet_manager.cc:1939] T a1760e1f6f544ef283b50d65b3f477f1 P 37da323536664cf2af59ba85eb42d4eb: Deleting consensus metadata
I20260809 07:48:27.408564 20552 catalog_manager.cc:4841] TS 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625): tablet a1760e1f6f544ef283b50d65b3f477f1 (table test-table [id=06c80ac20cbc4306bcfcd05a5b5a5854]) successfully deleted
Aug 09 07:48:27 dist-test-slave-10pc krb5kdc[20523](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261707, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 07:48:27 dist-test-slave-10pc krb5kdc[20523](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261707, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:27.433073 21211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:27.423901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.18.71.126:42059)
0809 07:48:27.424185 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:27.424195 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:27.424277 (+ 82us) server_negotiation.cc:407] Connection header received
0809 07:48:27.424498 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:27.424501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:27.424556 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:27.424651 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:27.425375 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:27.425940 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:27.426773 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:27.427010 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:27.427123 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:27.429749 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:27.429752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:27.429755 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:27.429778 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:27.431390 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:27.431759 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:27.431762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:27.431763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:27.431838 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:27.432091 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:27.432094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:27.432095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:27.432283 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:27.432394 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:27.432725 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 07:48:27.432835 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":101,"threads_started":1}
I20260809 07:48:27.435710 20567 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:27.436156 20567 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:27.440743 20719 tablet_service.cc:1467] Processing CreateTablet for tablet 78f5b9d99fc04f16a942e5d0f540e3f6 (DEFAULT_TABLE table=test-table [id=780aa93085174b4c90b0af4837ecde78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:27.440781 20852 tablet_service.cc:1467] Processing CreateTablet for tablet 78f5b9d99fc04f16a942e5d0f540e3f6 (DEFAULT_TABLE table=test-table [id=780aa93085174b4c90b0af4837ecde78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:27.440883 21012 tablet_service.cc:1467] Processing CreateTablet for tablet 78f5b9d99fc04f16a942e5d0f540e3f6 (DEFAULT_TABLE table=test-table [id=780aa93085174b4c90b0af4837ecde78]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:27.441074 20852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f5b9d99fc04f16a942e5d0f540e3f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:27.441107 20719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f5b9d99fc04f16a942e5d0f540e3f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:27.441126 21012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78f5b9d99fc04f16a942e5d0f540e3f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:27.443200 21212 tablet_bootstrap.cc:492] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: Bootstrap starting.
I20260809 07:48:27.443640 21213 tablet_bootstrap.cc:492] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Bootstrap starting.
I20260809 07:48:27.444424 21213 tablet_bootstrap.cc:654] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:27.444598 21212 tablet_bootstrap.cc:654] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:27.444696 21214 tablet_bootstrap.cc:492] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: Bootstrap starting.
I20260809 07:48:27.445595 21214 tablet_bootstrap.cc:654] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:27.445722 21213 tablet_bootstrap.cc:492] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: No bootstrap required, opened a new log
I20260809 07:48:27.445767 21212 tablet_bootstrap.cc:492] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: No bootstrap required, opened a new log
I20260809 07:48:27.445823 21213 ts_tablet_manager.cc:1397] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 07:48:27.445834 21212 ts_tablet_manager.cc:1397] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 07:48:27.446281 21213 raft_consensus.cc:348] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.446264 21212 raft_consensus.cc:348] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.446403 21213 raft_consensus.cc:374] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:27.446412 21212 raft_consensus.cc:374] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:27.446447 21213 raft_consensus.cc:729] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e4f0d1a8da4a1a9b26b502d82f02a6, State: Initialized, Role: FOLLOWER
I20260809 07:48:27.446493 21212 raft_consensus.cc:729] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37da323536664cf2af59ba85eb42d4eb, State: Initialized, Role: FOLLOWER
I20260809 07:48:27.446589 21213 consensus_queue.cc:260] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.446632 21212 consensus_queue.cc:260] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.446893 21213 ts_tablet_manager.cc:1428] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 07:48:27.446930 21212 ts_tablet_manager.cc:1428] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 07:48:27.448511 21214 tablet_bootstrap.cc:492] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: No bootstrap required, opened a new log
I20260809 07:48:27.448602 21214 ts_tablet_manager.cc:1397] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:48:27.449013 21214 raft_consensus.cc:348] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.449134 21214 raft_consensus.cc:374] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:27.449172 21214 raft_consensus.cc:729] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b55fde94664685a35cfaf15d8b8054, State: Initialized, Role: FOLLOWER
I20260809 07:48:27.449304 21214 consensus_queue.cc:260] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.449726 21214 ts_tablet_manager.cc:1428] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:27.522871 21219 raft_consensus.cc:482] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:27.523070 21219 raft_consensus.cc:504] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.523603 21219 leader_election.cc:288] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625), 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513)
I20260809 07:48:27.524050 20739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5b9d99fc04f16a942e5d0f540e3f6" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37da323536664cf2af59ba85eb42d4eb" is_pre_election: true
I20260809 07:48:27.524112 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5b9d99fc04f16a942e5d0f540e3f6" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" is_pre_election: true
I20260809 07:48:27.524245 20739 raft_consensus.cc:2454] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 0.
I20260809 07:48:27.524260 20886 raft_consensus.cc:2454] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 0.
I20260809 07:48:27.524627 20964 leader_election.cc:302] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98e4f0d1a8da4a1a9b26b502d82f02a6, d6b55fde94664685a35cfaf15d8b8054; no voters:
I20260809 07:48:27.524852 21219 raft_consensus.cc:2781] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:27.524922 21219 raft_consensus.cc:482] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:27.524964 21219 raft_consensus.cc:3037] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:27.525978 21219 raft_consensus.cc:504] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.526302 21219 leader_election.cc:288] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 election: Requested vote from peers 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625), 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513)
I20260809 07:48:27.526651 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5b9d99fc04f16a942e5d0f540e3f6" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6"
I20260809 07:48:27.526651 20739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78f5b9d99fc04f16a942e5d0f540e3f6" candidate_uuid: "d6b55fde94664685a35cfaf15d8b8054" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37da323536664cf2af59ba85eb42d4eb"
I20260809 07:48:27.526780 20886 raft_consensus.cc:3037] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:27.526780 20739 raft_consensus.cc:3037] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:27.527925 20739 raft_consensus.cc:2454] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 1.
I20260809 07:48:27.527925 20886 raft_consensus.cc:2454] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6b55fde94664685a35cfaf15d8b8054 in term 1.
I20260809 07:48:27.528204 20964 leader_election.cc:302] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98e4f0d1a8da4a1a9b26b502d82f02a6, d6b55fde94664685a35cfaf15d8b8054; no voters:
I20260809 07:48:27.528369 21219 raft_consensus.cc:2781] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:27.528643 21219 raft_consensus.cc:686] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 1 LEADER]: Becoming Leader. State: Replica: d6b55fde94664685a35cfaf15d8b8054, State: Running, Role: LEADER
I20260809 07:48:27.528808 21219 consensus_queue.cc:237] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.530479 20567 catalog_manager.cc:5495] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "d6b55fde94664685a35cfaf15d8b8054" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } health_report { overall_health: HEALTHY } } }
I20260809 07:48:27.550061 21223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:27.546794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55806 (local address 127.18.71.67:42025)
0809 07:48:27.547099 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:27.547106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:27.547118 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:48:27.547157 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:27.547158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:27.547198 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:27.547301 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:27.547561 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:27.548276 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:27.549144 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:27.549383 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:27.549465 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:27.549679 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:27.549746 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:27.549800 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:27.549879 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 07:48:27.549923 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":143,"threads_started":1}
I20260809 07:48:27.556145 20739 raft_consensus.cc:1261] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Refusing update from remote peer d6b55fde94664685a35cfaf15d8b8054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:27.556159 20886 raft_consensus.cc:1261] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Refusing update from remote peer d6b55fde94664685a35cfaf15d8b8054: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:27.556725 21220 consensus_queue.cc:1035] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:27.556870 21219 consensus_queue.cc:1035] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:27.588186 20569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:27.588362 20569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:27.590314 20569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Sending DeleteTablet for 3 replicas of tablet 78f5b9d99fc04f16a942e5d0f540e3f6
I20260809 07:48:27.590929 21012 tablet_service.cc:1514] Processing DeleteTablet for tablet 78f5b9d99fc04f16a942e5d0f540e3f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:27 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:43000
I20260809 07:48:27.590929 20719 tablet_service.cc:1514] Processing DeleteTablet for tablet 78f5b9d99fc04f16a942e5d0f540e3f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:27 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:57760
I20260809 07:48:27.591102 21203 tablet_replica.cc:330] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: stopping tablet replica
I20260809 07:48:27.591102 21201 tablet_replica.cc:330] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: stopping tablet replica
I20260809 07:48:27.591214 21201 raft_consensus.cc:2229] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:27.591226 21203 raft_consensus.cc:2229] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:27.591331 21201 raft_consensus.cc:2258] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:27.591280 20852 tablet_service.cc:1514] Processing DeleteTablet for tablet 78f5b9d99fc04f16a942e5d0f540e3f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:27 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:38398
I20260809 07:48:27.591403 21203 raft_consensus.cc:2258] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:27.591498 21202 tablet_replica.cc:330] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: stopping tablet replica
I20260809 07:48:27.591631 21202 raft_consensus.cc:2229] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:27.591751 21202 raft_consensus.cc:2258] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:27.592062 21201 ts_tablet_manager.cc:1905] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:27.592062 21203 ts_tablet_manager.cc:1905] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:27.592041 20569 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47782:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 07:48:27.592509 21202 ts_tablet_manager.cc:1905] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 07:48:27.592738 20569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:27.594684 21203 ts_tablet_manager.cc:1918] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:27.594688 21201 ts_tablet_manager.cc:1918] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:27.594765 21203 log.cc:1196] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/78f5b9d99fc04f16a942e5d0f540e3f6
I20260809 07:48:27.594765 21201 log.cc:1196] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/78f5b9d99fc04f16a942e5d0f540e3f6
I20260809 07:48:27.595039 21201 ts_tablet_manager.cc:1939] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 37da323536664cf2af59ba85eb42d4eb: Deleting consensus metadata
I20260809 07:48:27.595198 21202 ts_tablet_manager.cc:1918] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:27.595286 21202 log.cc:1196] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/78f5b9d99fc04f16a942e5d0f540e3f6
I20260809 07:48:27.595577 21202 ts_tablet_manager.cc:1939] T 78f5b9d99fc04f16a942e5d0f540e3f6 P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting consensus metadata
I20260809 07:48:27.595741 20552 catalog_manager.cc:4841] TS 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625): tablet 78f5b9d99fc04f16a942e5d0f540e3f6 (table test-table [id=780aa93085174b4c90b0af4837ecde78]) successfully deleted
I20260809 07:48:27.595039 21203 ts_tablet_manager.cc:1939] T 78f5b9d99fc04f16a942e5d0f540e3f6 P d6b55fde94664685a35cfaf15d8b8054: Deleting consensus metadata
I20260809 07:48:27.596285 20555 catalog_manager.cc:4841] TS 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513): tablet 78f5b9d99fc04f16a942e5d0f540e3f6 (table test-table [id=780aa93085174b4c90b0af4837ecde78]) successfully deleted
I20260809 07:48:27.596868 20554 catalog_manager.cc:4841] TS d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025): tablet 78f5b9d99fc04f16a942e5d0f540e3f6 (table test-table [id=780aa93085174b4c90b0af4837ecde78]) successfully deleted
I20260809 07:48:27.597605 20719 tablet_service.cc:1467] Processing CreateTablet for tablet 010073c5d1674422a8e73b974f67adcb (DEFAULT_TABLE table=test-table [id=a5a78456faa448aab938298c34970088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:27.597877 20719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010073c5d1674422a8e73b974f67adcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:27.597855 20852 tablet_service.cc:1467] Processing CreateTablet for tablet 010073c5d1674422a8e73b974f67adcb (DEFAULT_TABLE table=test-table [id=a5a78456faa448aab938298c34970088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:27.598073 20852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010073c5d1674422a8e73b974f67adcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:27.600072 21212 tablet_bootstrap.cc:492] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb: Bootstrap starting.
I20260809 07:48:27.600133 21213 tablet_bootstrap.cc:492] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: Bootstrap starting.
I20260809 07:48:27.600229 21012 tablet_service.cc:1467] Processing CreateTablet for tablet 010073c5d1674422a8e73b974f67adcb (DEFAULT_TABLE table=test-table [id=a5a78456faa448aab938298c34970088]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:27.600525 21012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 010073c5d1674422a8e73b974f67adcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:27.600945 21212 tablet_bootstrap.cc:654] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:27.600946 21213 tablet_bootstrap.cc:654] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:27.602226 21213 tablet_bootstrap.cc:492] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: No bootstrap required, opened a new log
I20260809 07:48:27.602345 21213 ts_tablet_manager.cc:1397] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:27.602814 21213 raft_consensus.cc:348] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.602902 21214 tablet_bootstrap.cc:492] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: Bootstrap starting.
I20260809 07:48:27.602946 21213 raft_consensus.cc:374] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:27.602985 21213 raft_consensus.cc:729] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e4f0d1a8da4a1a9b26b502d82f02a6, State: Initialized, Role: FOLLOWER
I20260809 07:48:27.603116 21213 consensus_queue.cc:260] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.603410 21213 ts_tablet_manager.cc:1428] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:27.603731 21214 tablet_bootstrap.cc:654] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:27.603875 21212 tablet_bootstrap.cc:492] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb: No bootstrap required, opened a new log
I20260809 07:48:27.603961 21212 ts_tablet_manager.cc:1397] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 07:48:27.604341 21212 raft_consensus.cc:348] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.604504 21212 raft_consensus.cc:374] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:27.604542 21212 raft_consensus.cc:729] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37da323536664cf2af59ba85eb42d4eb, State: Initialized, Role: FOLLOWER
I20260809 07:48:27.604681 21212 consensus_queue.cc:260] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.604729 21214 tablet_bootstrap.cc:492] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: No bootstrap required, opened a new log
I20260809 07:48:27.604815 21214 ts_tablet_manager.cc:1397] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:27.604965 21212 ts_tablet_manager.cc:1428] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:27.605144 21214 raft_consensus.cc:348] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.605222 21214 raft_consensus.cc:374] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:27.605243 21214 raft_consensus.cc:729] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6b55fde94664685a35cfaf15d8b8054, State: Initialized, Role: FOLLOWER
I20260809 07:48:27.605324 21214 consensus_queue.cc:260] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.605594 21214 ts_tablet_manager.cc:1428] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:27.613920 21195 raft_consensus.cc:482] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:27.614063 21195 raft_consensus.cc:504] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.614441 21195 leader_election.cc:288] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513), d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025)
I20260809 07:48:27.614776 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010073c5d1674422a8e73b974f67adcb" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" is_pre_election: true
I20260809 07:48:27.614796 21031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010073c5d1674422a8e73b974f67adcb" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b55fde94664685a35cfaf15d8b8054" is_pre_election: true
I20260809 07:48:27.614914 21031 raft_consensus.cc:2454] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 0.
I20260809 07:48:27.614914 20886 raft_consensus.cc:2454] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 0.
I20260809 07:48:27.615190 20666 leader_election.cc:302] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37da323536664cf2af59ba85eb42d4eb, 98e4f0d1a8da4a1a9b26b502d82f02a6; no voters:
I20260809 07:48:27.615362 21195 raft_consensus.cc:2781] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:27.615423 21195 raft_consensus.cc:482] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:27.615464 21195 raft_consensus.cc:3037] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:27.616430 21195 raft_consensus.cc:504] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.616806 21195 leader_election.cc:288] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 election: Requested vote from peers 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513), d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025)
I20260809 07:48:27.617069 20886 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010073c5d1674422a8e73b974f67adcb" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6"
I20260809 07:48:27.617218 20886 raft_consensus.cc:3037] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:27.617199 21031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "010073c5d1674422a8e73b974f67adcb" candidate_uuid: "37da323536664cf2af59ba85eb42d4eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6b55fde94664685a35cfaf15d8b8054"
I20260809 07:48:27.617302 21031 raft_consensus.cc:3037] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:27.618361 21031 raft_consensus.cc:2454] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 1.
I20260809 07:48:27.618361 20886 raft_consensus.cc:2454] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37da323536664cf2af59ba85eb42d4eb in term 1.
I20260809 07:48:27.618644 20665 leader_election.cc:302] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37da323536664cf2af59ba85eb42d4eb, d6b55fde94664685a35cfaf15d8b8054; no voters:
I20260809 07:48:27.618857 21195 raft_consensus.cc:2781] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:27.618947 21195 raft_consensus.cc:686] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 1 LEADER]: Becoming Leader. State: Replica: 37da323536664cf2af59ba85eb42d4eb, State: Running, Role: LEADER
I20260809 07:48:27.619089 21195 consensus_queue.cc:237] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } }
I20260809 07:48:27.620739 20569 catalog_manager.cc:5495] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb reported cstate change: term changed from 0 to 1, leader changed from <none> to 37da323536664cf2af59ba85eb42d4eb (127.18.71.65). New cstate: current_term: 1 leader_uuid: "37da323536664cf2af59ba85eb42d4eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37da323536664cf2af59ba85eb42d4eb" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 35625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 } health_report { overall_health: UNKNOWN } } }
I20260809 07:48:27.635901 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:27.632811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.18.71.65:35625)
0809 07:48:27.633102 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:27.633127 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:27.633138 (+ 11us) server_negotiation.cc:407] Connection header received
0809 07:48:27.633176 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:27.633178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:27.633228 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:27.633296 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:27.633470 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:27.633958 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:27.634802 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:27.635032 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:27.635181 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:27.635441 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:27.635510 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:27.635609 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:27.635693 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 07:48:27.635744 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":152,"threads_started":1}
I20260809 07:48:27.641597 21031 raft_consensus.cc:1261] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Refusing update from remote peer 37da323536664cf2af59ba85eb42d4eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:27.641664 20886 raft_consensus.cc:1261] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Refusing update from remote peer 37da323536664cf2af59ba85eb42d4eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:27.641999 21176 consensus_queue.cc:1035] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6b55fde94664685a35cfaf15d8b8054" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:27.642133 21195 consensus_queue.cc:1035] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "98e4f0d1a8da4a1a9b26b502d82f02a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:48:27.654675 21180 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 07:48:29.202241 20569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:29.202443 20569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47782:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:29.204906 20569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 601cb7bd3cdc4affb6ad824debceba79: Sending DeleteTablet for 3 replicas of tablet 010073c5d1674422a8e73b974f67adcb
I20260809 07:48:29.205508 20852 tablet_service.cc:1514] Processing DeleteTablet for tablet 010073c5d1674422a8e73b974f67adcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:29 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:38398
I20260809 07:48:29.205516 20719 tablet_service.cc:1514] Processing DeleteTablet for tablet 010073c5d1674422a8e73b974f67adcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:29 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:57760
I20260809 07:48:29.205539 21012 tablet_service.cc:1514] Processing DeleteTablet for tablet 010073c5d1674422a8e73b974f67adcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:29 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:43000
I20260809 07:48:29.205859 21262 tablet_replica.cc:330] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: stopping tablet replica
I20260809 07:48:29.205896 21263 tablet_replica.cc:330] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: stopping tablet replica
I20260809 07:48:29.205896 21261 tablet_replica.cc:330] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb: stopping tablet replica
I20260809 07:48:29.206032 21263 raft_consensus.cc:2229] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:29.206063 21261 raft_consensus.cc:2229] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:29.206151 21263 raft_consensus.cc:2258] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:29.206067 21262 raft_consensus.cc:2229] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:29.206221 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 20648
I20260809 07:48:29.206235 21261 raft_consensus.cc:2258] T 010073c5d1674422a8e73b974f67adcb P 37da323536664cf2af59ba85eb42d4eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:29.208323 21263 ts_tablet_manager.cc:1905] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:29.208549 21262 raft_consensus.cc:2258] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:29.209236 21262 ts_tablet_manager.cc:1905] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:29.211971 21262 ts_tablet_manager.cc:1918] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 07:48:29.212038 21262 log.cc:1196] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/010073c5d1674422a8e73b974f67adcb
I20260809 07:48:29.212319 21262 ts_tablet_manager.cc:1939] T 010073c5d1674422a8e73b974f67adcb P 98e4f0d1a8da4a1a9b26b502d82f02a6: Deleting consensus metadata
I20260809 07:48:29.213034 20555 catalog_manager.cc:4841] TS 98e4f0d1a8da4a1a9b26b502d82f02a6 (127.18.71.66:39513): tablet 010073c5d1674422a8e73b974f67adcb (table test-table [id=a5a78456faa448aab938298c34970088]) successfully deleted
I20260809 07:48:29.214931 21263 ts_tablet_manager.cc:1918] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 07:48:29.215014 21263 log.cc:1196] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/010073c5d1674422a8e73b974f67adcb
I20260809 07:48:29.215368 21263 ts_tablet_manager.cc:1939] T 010073c5d1674422a8e73b974f67adcb P d6b55fde94664685a35cfaf15d8b8054: Deleting consensus metadata
I20260809 07:48:29.216064 20554 catalog_manager.cc:4841] TS d6b55fde94664685a35cfaf15d8b8054 (127.18.71.67:42025): tablet 010073c5d1674422a8e73b974f67adcb (table test-table [id=a5a78456faa448aab938298c34970088]) successfully deleted
W20260809 07:48:29.218406 20552 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:35625 (error 108)
I20260809 07:48:29.219267 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 20799
I20260809 07:48:29.220399 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:29.219155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:35625 (local address 127.0.0.1:44470)
0809 07:48:29.220201 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:29.220294 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:35625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
W20260809 07:48:29.220551 20552 catalog_manager.cc:4584] TS 37da323536664cf2af59ba85eb42d4eb (127.18.71.65:35625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 010073c5d1674422a8e73b974f67adcb: Network error: Client connection negotiation failed: client connection to 127.18.71.65:35625: connect: Connection refused (error 111)
I20260809 07:48:29.229714 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 20945
I20260809 07:48:29.238173 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 20536
2026-08-09T07:48:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6151 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:29 dist-test-slave-10pc krb5kdc[21268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:29 dist-test-slave-10pc krb5kdc[21268](info): set up 2 sockets
Aug 09 07:48:29 dist-test-slave-10pc krb5kdc[21268](info): commencing operation
krb5kdc: starting...
W20260809 07:48:30.336562 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:30 dist-test-slave-10pc krb5kdc[21268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:30.468295 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45983
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:45983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:30.579864 21282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:30.580164 21282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:30.580240 21282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:30.584230 21282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:30.584340 21282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:30.584381 21282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:30.584420 21282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:30.584474 21282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:30.589984 21282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:45983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:30.591147 21282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:30.592226 21282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:30.600834 21287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:30.600945 21282 server_base.cc:1034] running on GCE node
W20260809 07:48:30.600803 21288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:30.600853 21290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:30.601624 21282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:30.602726 21282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:30.603890 21282 hybrid_clock.cc:648] HybridClock initialized: now 1786261710603860 us; error 40 us; skew 500 ppm
Aug 09 07:48:30 dist-test-slave-10pc krb5kdc[21268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261710, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:30.611135 21282 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:30.612118 21282 webserver.cc:460] Webserver started at http://127.18.71.126:43455/ using document root <none> and password file <none>
I20260809 07:48:30.612777 21282 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:30.612843 21282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:30.613092 21282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:30.615144 21282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "9a9060b5eb654f33a4e85fd87e0d376d"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-10pc"
server_key: "de6e92f393b671ff9eed98074aad783b"
server_key_iv: "d6f4e9a80f98fd96029c31ba5bb34a67"
server_key_version: "encryptionkey@0"
I20260809 07:48:30.615731 21282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "9a9060b5eb654f33a4e85fd87e0d376d"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-10pc"
server_key: "de6e92f393b671ff9eed98074aad783b"
server_key_iv: "d6f4e9a80f98fd96029c31ba5bb34a67"
server_key_version: "encryptionkey@0"
I20260809 07:48:30.619468 21282 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260809 07:48:30.621817 21297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:30.623047 21282 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:30.623200 21282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "9a9060b5eb654f33a4e85fd87e0d376d"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-10pc"
server_key: "de6e92f393b671ff9eed98074aad783b"
server_key_iv: "d6f4e9a80f98fd96029c31ba5bb34a67"
server_key_version: "encryptionkey@0"
I20260809 07:48:30.623311 21282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:30.650188 21282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:30.653200 21282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:30.653412 21282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:30.660578 21282 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:45983
I20260809 07:48:30.660593 21352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:45983 every 8 connection(s)
I20260809 07:48:30.661543 21282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:30.664285 21353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:30.664640 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 21282
I20260809 07:48:30.664727 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:30.665024 18717 external_mini_cluster.cc:1428] Setting key f444b8d9b99c5bd5b4c7b22d60875211
I20260809 07:48:30.671613 21353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d: Bootstrap starting.
I20260809 07:48:30.674250 21353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d: Neither blocks nor log segments found. Creating new log.
Aug 09 07:48:30 dist-test-slave-10pc krb5kdc[21268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:30.675186 21353 log.cc:824] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:30.677141 21353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d: No bootstrap required, opened a new log
I20260809 07:48:30.680153 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:30.666416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.18.71.126:45983)
0809 07:48:30.666860 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:30.666879 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:30.666921 (+ 42us) server_negotiation.cc:407] Connection header received
0809 07:48:30.667747 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:30.667773 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:30.668200 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:30.668594 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:30.669368 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:30.670934 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:30.671813 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:30.672487 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:30.672650 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:30.675533 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:30.675539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:30.675553 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:30.675580 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:30.677438 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:30.677920 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:30.677925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:30.677929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:30.678041 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:30.678324 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:30.678326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:30.678327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:30.678712 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:30.678862 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:30.679154 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 07:48:30.679429 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":301,"thread_start_us":107,"threads_started":1}
I20260809 07:48:30.681221 21353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45983 } }
I20260809 07:48:30.681491 21353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:30.681556 21353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9060b5eb654f33a4e85fd87e0d376d, State: Initialized, Role: FOLLOWER
I20260809 07:48:30.682018 21353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45983 } }
I20260809 07:48:30.682199 21353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:30.682286 21353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:30.682444 21353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:30.683306 21353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45983 } }
I20260809 07:48:30.683641 21353 leader_election.cc:302] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a9060b5eb654f33a4e85fd87e0d376d; no voters:
I20260809 07:48:30.683938 21353 leader_election.cc:288] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:30.684147 21358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:30.684450 21358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [term 1 LEADER]: Becoming Leader. State: Replica: 9a9060b5eb654f33a4e85fd87e0d376d, State: Running, Role: LEADER
I20260809 07:48:30.684795 21358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45983 } }
I20260809 07:48:30.685204 21353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:30.686379 21359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45983 } } }
I20260809 07:48:30.686556 21359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:30.686854 21360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a9060b5eb654f33a4e85fd87e0d376d. Latest consensus state: current_term: 1 leader_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a9060b5eb654f33a4e85fd87e0d376d" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45983 } } }
I20260809 07:48:30.686951 21360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:30.687606 21363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:30.690351 21363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:30.695793 21363 catalog_manager.cc:1347] Generated new cluster ID: 56669d1ff03f401a9bc87fc5e2c1c490
I20260809 07:48:30.695861 21363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:30.721592 21363 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:30.722770 21363 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:30.726780 21363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a9060b5eb654f33a4e85fd87e0d376d: Generated new TSK 0
I20260809 07:48:30.727445 21363 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:30.767601 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45983
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:30.906215 21380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:30.906452 21380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:30.906493 21380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:30.910355 21380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:30.910410 21380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:30.910467 21380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:30.915586 21380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:30.916563 21380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:30.917520 21380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:30.927174 21385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:30.927119 21386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:30.927189 21388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:30.927393 21380 server_base.cc:1034] running on GCE node
I20260809 07:48:30.927927 21380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:30.928603 21380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:30.929764 21380 hybrid_clock.cc:648] HybridClock initialized: now 1786261710929748 us; error 29 us; skew 500 ppm
Aug 09 07:48:30 dist-test-slave-10pc krb5kdc[21268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261710, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:30.932490 21380 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:30.933290 21380 webserver.cc:460] Webserver started at http://127.18.71.65:44841/ using document root <none> and password file <none>
I20260809 07:48:30.933801 21380 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:30.933858 21380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:30.934020 21380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:30.936014 21380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "addda4c218fd4f91bd5dfdf05e072252"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-10pc"
server_key: "70559b2843bd82637f2ea1a0396e224c"
server_key_iv: "51b43b3161201165624d324bf5052b89"
server_key_version: "encryptionkey@0"
I20260809 07:48:30.936610 21380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "addda4c218fd4f91bd5dfdf05e072252"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-10pc"
server_key: "70559b2843bd82637f2ea1a0396e224c"
server_key_iv: "51b43b3161201165624d324bf5052b89"
server_key_version: "encryptionkey@0"
I20260809 07:48:30.940132 21380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:30.942569 21395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:30.943837 21380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:30.943969 21380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "addda4c218fd4f91bd5dfdf05e072252"
format_stamp: "Formatted at 2026-08-09 07:48:30 on dist-test-slave-10pc"
server_key: "70559b2843bd82637f2ea1a0396e224c"
server_key_iv: "51b43b3161201165624d324bf5052b89"
server_key_version: "encryptionkey@0"
I20260809 07:48:30.944077 21380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:30.964217 21380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:30.967082 21380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:30.967298 21380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:30.967885 21380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:30.968765 21380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:30.968834 21380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:30.968905 21380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:30.968950 21380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:30.977761 21380 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:36039
I20260809 07:48:30.977787 21517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:36039 every 8 connection(s)
I20260809 07:48:30.978601 21380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:30.984920 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 21380
I20260809 07:48:30.985057 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:30.985335 18717 external_mini_cluster.cc:1428] Setting key 5a7fb1026997a84955048b8a13440866
Aug 09 07:48:30 dist-test-slave-10pc krb5kdc[21268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261710, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:30.992735 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:30.980403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:58665 (local address 127.18.71.126:45983)
0809 07:48:30.980591 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:30.980596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:30.981349 (+ 753us) server_negotiation.cc:407] Connection header received
0809 07:48:30.982583 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:30.982585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:30.982633 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:30.982711 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:30.983848 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:30.984374 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:30.985805 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:30.986158 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:30.986284 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:30.989195 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:30.989198 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:30.989201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:30.989227 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:30.990635 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:30.991155 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:30.991158 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:30.991160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:30.991249 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:30.991603 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:30.991606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:30.991608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:30.991798 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:30.991906 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:30.992402 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 07:48:30.992560 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":91}
I20260809 07:48:30.993499 21520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:30.980762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45983 (local address 127.18.71.65:58665)
0809 07:48:30.981199 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:30.981241 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:30.982383 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:30.982886 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:30.982892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:30.983237 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:30.983722 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:30.983736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:30.984546 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:30.984548 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:30.985668 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:30.985679 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:30.986315 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:30.986319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:30.986549 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:30.987252 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:30.987268 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:30.989020 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:30.990753 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:30.990759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:30.990773 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:30.991043 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:30.991339 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:30.991343 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:30.991345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:30.991488 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:30.991905 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:30.991914 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:30.992159 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 07:48:30.992336 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 07:48:30.992679 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260809 07:48:30.994719 21518 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45983
I20260809 07:48:30.994971 21518 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:30.995491 21518 heartbeater.cc:511] Master 127.18.71.126:45983 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:30.997087 21316 ts_manager.cc:194] Registered new tserver with Master: addda4c218fd4f91bd5dfdf05e072252 (127.18.71.65:36039)
I20260809 07:48:30.998363 21316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:58665
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:31.014912 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45983
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:31.129259 21523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:31.129626 21523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:31.129705 21523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:31.134966 21523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:31.135053 21523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:31.135175 21523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:31.141089 21523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:31.142097 21523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:31.143075 21523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:31.153021 21529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:31.153100 21528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:31.153033 21532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:31.153298 21523 server_base.cc:1034] running on GCE node
I20260809 07:48:31.153786 21523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:31.154436 21523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:31.155618 21523 hybrid_clock.cc:648] HybridClock initialized: now 1786261711155598 us; error 28 us; skew 500 ppm
Aug 09 07:48:31 dist-test-slave-10pc krb5kdc[21268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261711, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:31.158603 21523 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:31.159493 21523 webserver.cc:460] Webserver started at http://127.18.71.66:43497/ using document root <none> and password file <none>
I20260809 07:48:31.160054 21523 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:31.160116 21523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:31.160370 21523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:31.162832 21523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "2977ce23e95a4baa86eb2dce7b7d747a"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-10pc"
server_key: "3a7ab074beb7f1c43d009e7398d8e382"
server_key_iv: "17f77da42ba90bedba3c9dc387a31f48"
server_key_version: "encryptionkey@0"
I20260809 07:48:31.163501 21523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "2977ce23e95a4baa86eb2dce7b7d747a"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-10pc"
server_key: "3a7ab074beb7f1c43d009e7398d8e382"
server_key_iv: "17f77da42ba90bedba3c9dc387a31f48"
server_key_version: "encryptionkey@0"
I20260809 07:48:31.167303 21523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:48:31.169778 21540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:31.171054 21523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:31.171203 21523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "2977ce23e95a4baa86eb2dce7b7d747a"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-10pc"
server_key: "3a7ab074beb7f1c43d009e7398d8e382"
server_key_iv: "17f77da42ba90bedba3c9dc387a31f48"
server_key_version: "encryptionkey@0"
I20260809 07:48:31.171322 21523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:31.192602 21523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:31.195472 21523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:31.195693 21523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:31.196349 21523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:31.197248 21523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:31.197321 21523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:31.197391 21523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:31.197460 21523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:31.206991 21523 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:33881
I20260809 07:48:31.207017 21658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:33881 every 8 connection(s)
I20260809 07:48:31.207938 21523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:31.216614 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 21523
I20260809 07:48:31.216746 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:31.217024 18717 external_mini_cluster.cc:1428] Setting key 10509a5e949ddbee172ab459b2f2c9a8
Aug 09 07:48:31 dist-test-slave-10pc krb5kdc[21268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261711, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:31.222401 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:31.209841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:45443 (local address 127.18.71.126:45983)
0809 07:48:31.209978 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:31.209982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:31.210791 (+ 809us) server_negotiation.cc:407] Connection header received
0809 07:48:31.212206 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:31.212209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:31.212270 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:31.212377 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:31.213598 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.214144 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.215341 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.215574 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.215696 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:31.218657 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:31.218660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:31.218662 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:31.218685 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:31.220108 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:31.220794 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:31.220797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:31.220799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:31.220880 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:31.221295 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:31.221297 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:31.221299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:31.221463 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:31.221536 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:31.222188 (+ 652us) server_negotiation.cc:299] Negotiation successful
0809 07:48:31.222287 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:48:31.223637 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:31.210145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45983 (local address 127.18.71.66:45443)
0809 07:48:31.210628 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:31.210677 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:31.211963 (+ 1286us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:31.212593 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:31.212600 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:31.212960 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:31.213449 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:31.213462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.214280 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.214284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:31.215212 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:31.215222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.215740 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.215744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:31.215942 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:31.216710 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:31.216727 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:31.218485 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:31.220232 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:31.220238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:31.220252 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:31.220669 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:31.220987 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:31.220993 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:31.220996 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:31.221194 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:31.221566 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:31.221574 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:31.221902 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 07:48:31.222131 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 07:48:31.222503 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":114,"threads_started":1}
I20260809 07:48:31.224933 21659 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45983
I20260809 07:48:31.225191 21659 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:31.225673 21659 heartbeater.cc:511] Master 127.18.71.126:45983 requested a full tablet report, sending...
I20260809 07:48:31.226732 21316 ts_manager.cc:194] Registered new tserver with Master: 2977ce23e95a4baa86eb2dce7b7d747a (127.18.71.66:33881)
I20260809 07:48:31.227367 21316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:45443
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:31.247061 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45983
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:31.367316 21664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:31.367575 21664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:31.367619 21664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:31.371579 21664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:31.371635 21664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:31.371691 21664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:31.376978 21664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:31.377938 21664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:31.378922 21664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:31.388842 21669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:31.388875 21671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:31.388857 21673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:31.389112 21664 server_base.cc:1034] running on GCE node
I20260809 07:48:31.389501 21664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:31.390147 21664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:31.391312 21664 hybrid_clock.cc:648] HybridClock initialized: now 1786261711391291 us; error 30 us; skew 500 ppm
Aug 09 07:48:31 dist-test-slave-10pc krb5kdc[21268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261711, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:31.394200 21664 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:31.395006 21664 webserver.cc:460] Webserver started at http://127.18.71.67:40291/ using document root <none> and password file <none>
I20260809 07:48:31.395526 21664 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:31.395584 21664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:31.395749 21664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:31.397959 21664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "c2125b2bc03742b592c50e47d7d90adb"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-10pc"
server_key: "699e2ba0e8394617630df31d11c83344"
server_key_iv: "1badebeedcbd555e6f6f4d8754b185bf"
server_key_version: "encryptionkey@0"
I20260809 07:48:31.398519 21664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "c2125b2bc03742b592c50e47d7d90adb"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-10pc"
server_key: "699e2ba0e8394617630df31d11c83344"
server_key_iv: "1badebeedcbd555e6f6f4d8754b185bf"
server_key_version: "encryptionkey@0"
I20260809 07:48:31.402910 21664 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:48:31.405428 21681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:31.406714 21664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:31.406833 21664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "c2125b2bc03742b592c50e47d7d90adb"
format_stamp: "Formatted at 2026-08-09 07:48:31 on dist-test-slave-10pc"
server_key: "699e2ba0e8394617630df31d11c83344"
server_key_iv: "1badebeedcbd555e6f6f4d8754b185bf"
server_key_version: "encryptionkey@0"
I20260809 07:48:31.407008 21664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:31.419596 21664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:31.422853 21664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:31.423018 21664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:31.423621 21664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:31.424441 21664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:31.424564 21664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:31.424631 21664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:31.424680 21664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:31.434492 21664 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:40341
I20260809 07:48:31.434528 21799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:40341 every 8 connection(s)
I20260809 07:48:31.435786 21664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:31.444043 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 21664
I20260809 07:48:31.444185 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:31.444604 18717 external_mini_cluster.cc:1428] Setting key 43b4018ac2136c3d4927d9373be2196e
Aug 09 07:48:31 dist-test-slave-10pc krb5kdc[21268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261711, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:31.449882 21356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:31.437620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:38751 (local address 127.18.71.126:45983)
0809 07:48:31.437765 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:31.437768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:31.438601 (+ 833us) server_negotiation.cc:407] Connection header received
0809 07:48:31.439908 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:31.439911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:31.439969 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:31.440048 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:31.441372 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.441920 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.443083 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.443321 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.443439 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:31.446373 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:31.446375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:31.446377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:31.446401 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:31.447907 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:31.448494 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:31.448499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:31.448501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:31.448617 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:31.448960 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:31.448962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:31.448963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:31.449137 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:31.449216 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:31.449670 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 07:48:31.449767 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:48:31.450747 21802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:31.437992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45983 (local address 127.18.71.67:38751)
0809 07:48:31.438440 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:31.438483 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:31.439695 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:31.440232 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:31.440239 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:31.440740 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:31.441227 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:31.441240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.442077 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.442080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:31.442954 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:31.442963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:31.443481 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:31.443484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:31.443654 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:31.444286 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:31.444308 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:31.446225 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:31.448009 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:31.448014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:31.448025 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:31.448283 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:31.448719 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:31.448735 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:31.448737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:31.448873 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:31.449229 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:31.449234 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:31.449458 (+ 224us) client_negotiation.cc:769] Sending connection context
0809 07:48:31.449614 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 07:48:31.449872 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":112,"threads_started":1}
I20260809 07:48:31.451900 21800 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45983
I20260809 07:48:31.452114 21800 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:31.452643 21800 heartbeater.cc:511] Master 127.18.71.126:45983 requested a full tablet report, sending...
I20260809 07:48:31.453647 21316 ts_manager.cc:194] Registered new tserver with Master: c2125b2bc03742b592c50e47d7d90adb (127.18.71.67:40341)
I20260809 07:48:31.454278 21316 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:38751
I20260809 07:48:31.458513 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 07:48:31.473342 21356 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:31.471841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37428 (local address 127.18.71.126:45983)
0809 07:48:31.471973 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:31.471977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:31.472168 (+ 191us) server_negotiation.cc:407] Connection header received
0809 07:48:31.472279 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:31.472281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:31.472321 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:31.472399 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:31.473174 (+ 775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37428: BlockingRecv error: recv got EOF from 127.0.0.1:37428 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:48:31.474529 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 21380
I20260809 07:48:31.486815 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 21523
I20260809 07:48:31.494088 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 21664
I20260809 07:48:31.502864 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 21282
2026-08-09T07:48:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2263 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:32 dist-test-slave-10pc krb5kdc[21813](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:32 dist-test-slave-10pc krb5kdc[21813](info): set up 2 sockets
Aug 09 07:48:32 dist-test-slave-10pc krb5kdc[21813](info): commencing operation
krb5kdc: starting...
W20260809 07:48:33.891860 18717 mini_kdc.cc:121] Time spent starting KDC: real 2.366s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:33 dist-test-slave-10pc krb5kdc[21813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:34.020507 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:38727
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:43983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:38727
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:34.138198 21827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:34.138500 21827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:34.138576 21827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:34.142601 21827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:34.142699 21827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:34.142735 21827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:34.142776 21827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:34.142805 21827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:34.148137 21827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:43983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:38727
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:38727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:34.149343 21827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:34.150411 21827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:34.159099 21832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:34.159140 21833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:34.159106 21835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:34.159310 21827 server_base.cc:1034] running on GCE node
I20260809 07:48:34.159929 21827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:34.161096 21827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:34.162299 21827 hybrid_clock.cc:648] HybridClock initialized: now 1786261714162286 us; error 29 us; skew 500 ppm
Aug 09 07:48:34 dist-test-slave-10pc krb5kdc[21813](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261714, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:34.165078 21827 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:34.165944 21827 webserver.cc:460] Webserver started at http://127.18.71.126:34151/ using document root <none> and password file <none>
I20260809 07:48:34.166544 21827 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:34.166636 21827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:34.166898 21827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:34.169004 21827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "5d8ea0c5cb564e5b8126a0bb32295c79"
format_stamp: "Formatted at 2026-08-09 07:48:34 on dist-test-slave-10pc"
server_key: "61fa6656977aeceab0764a31c9c50511"
server_key_iv: "cb73a0221e9bb0509d3a9c743494ce73"
server_key_version: "encryptionkey@0"
I20260809 07:48:34.169591 21827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "5d8ea0c5cb564e5b8126a0bb32295c79"
format_stamp: "Formatted at 2026-08-09 07:48:34 on dist-test-slave-10pc"
server_key: "61fa6656977aeceab0764a31c9c50511"
server_key_iv: "cb73a0221e9bb0509d3a9c743494ce73"
server_key_version: "encryptionkey@0"
I20260809 07:48:34.173321 21827 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:34.175696 21843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:34.176971 21827 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:34.177109 21827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "5d8ea0c5cb564e5b8126a0bb32295c79"
format_stamp: "Formatted at 2026-08-09 07:48:34 on dist-test-slave-10pc"
server_key: "61fa6656977aeceab0764a31c9c50511"
server_key_iv: "cb73a0221e9bb0509d3a9c743494ce73"
server_key_version: "encryptionkey@0"
I20260809 07:48:34.177225 21827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:34.186256 21827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:34.189245 21827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:34.189445 21827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:34.197054 21827 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:38727
I20260809 07:48:34.197435 21896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:38727 every 8 connection(s)
I20260809 07:48:34.198050 21827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:34.200932 21898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:34.206502 21898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79: Bootstrap starting.
I20260809 07:48:34.207832 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 21827
I20260809 07:48:34.207955 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:34.208238 18717 external_mini_cluster.cc:1428] Setting key 4bd04c7cbd50c6c09a5c601be3ef2f3b
I20260809 07:48:34.209569 21898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:34.210350 21898 log.cc:824] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:34.212877 21898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79: No bootstrap required, opened a new log
I20260809 07:48:34.217509 21898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38727 } }
I20260809 07:48:34.217763 21898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:34.217846 21898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d8ea0c5cb564e5b8126a0bb32295c79, State: Initialized, Role: FOLLOWER
Aug 09 07:48:34 dist-test-slave-10pc krb5kdc[21813](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:34.218398 21898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38727 } }
I20260809 07:48:34.218526 21898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:34.218608 21898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:34.218703 21898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:34.219632 21898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38727 } }
I20260809 07:48:34.219986 21898 leader_election.cc:302] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5d8ea0c5cb564e5b8126a0bb32295c79; no voters:
I20260809 07:48:34.220335 21898 leader_election.cc:288] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:34.220407 21903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:34.220654 21903 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [term 1 LEADER]: Becoming Leader. State: Replica: 5d8ea0c5cb564e5b8126a0bb32295c79, State: Running, Role: LEADER
I20260809 07:48:34.221011 21903 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38727 } }
I20260809 07:48:34.221712 21898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:34.222757 21906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5d8ea0c5cb564e5b8126a0bb32295c79. Latest consensus state: current_term: 1 leader_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38727 } } }
I20260809 07:48:34.222892 21906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:34.223269 21904 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5d8ea0c5cb564e5b8126a0bb32295c79" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38727 } } }
I20260809 07:48:34.223380 21904 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:34.225023 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:34.209376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55232 (local address 127.18.71.126:38727)
0809 07:48:34.209791 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:34.209808 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:34.209843 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:48:34.210590 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:34.210610 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:34.211035 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:34.211364 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:34.212324 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:34.213706 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:34.214991 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:34.215505 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:34.215642 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:34.218810 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:34.218815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:34.218828 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:34.218855 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:34.220880 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:34.221402 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:34.221406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:34.221411 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:34.221517 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:34.221862 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:34.221865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:34.221866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:34.222427 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:34.223298 (+ 871us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:34.223476 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 07:48:34.223784 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":133,"threads_started":1}
W20260809 07:48:34.226966 21922 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:34.227068 21922 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:34.227136 21915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:34.227931 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:48:34.227968 21915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:34.234421 21915 catalog_manager.cc:1347] Generated new cluster ID: a676d84514fe4bfda36d2441928c86cd
I20260809 07:48:34.234494 21915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:34.242576 21915 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:34.244720 21915 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:34.251149 21901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:34.245213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55242 (local address 127.18.71.126:38727)
0809 07:48:34.245318 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:34.245322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:34.245924 (+ 602us) server_negotiation.cc:407] Connection header received
0809 07:48:34.246097 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:34.246100 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:34.246141 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:34.246188 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:34.246821 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:34.247578 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:34.248791 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:34.249164 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:34.249284 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:34.250194 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:34.250198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:34.250201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:34.250728 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:34.250760 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:34.250779 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:34.250872 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:34.250954 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 07:48:34.251012 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 07:48:34.253300 21915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5d8ea0c5cb564e5b8126a0bb32295c79: Generated new TSK 0
I20260809 07:48:34.253934 21915 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:48:34.261029 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 21827
2026-08-09T07:48:34Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2757 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:34 dist-test-slave-10pc krb5kdc[21936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:34 dist-test-slave-10pc krb5kdc[21936](info): set up 2 sockets
Aug 09 07:48:34 dist-test-slave-10pc krb5kdc[21936](info): commencing operation
krb5kdc: starting...
W20260809 07:48:35.357088 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:35 dist-test-slave-10pc krb5kdc[21936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:35.492269 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46821
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:46821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:35.606984 21950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:35.607331 21950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:35.607416 21950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:35.611554 21950 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:35.611634 21950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:35.611672 21950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:35.611728 21950 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:35.611765 21950 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:35.617192 21950 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:46821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:35.618333 21950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:35.619408 21950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:35.628106 21956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:35.628101 21955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:35.628098 21958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:35.628476 21950 server_base.cc:1034] running on GCE node
I20260809 07:48:35.629041 21950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:35.630187 21950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:35.631350 21950 hybrid_clock.cc:648] HybridClock initialized: now 1786261715631330 us; error 31 us; skew 500 ppm
Aug 09 07:48:35 dist-test-slave-10pc krb5kdc[21936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:35.638234 21950 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:35.639197 21950 webserver.cc:460] Webserver started at http://127.18.71.126:41387/ using document root <none> and password file <none>
I20260809 07:48:35.639781 21950 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:35.639842 21950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:35.640110 21950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:35.642220 21950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83"
format_stamp: "Formatted at 2026-08-09 07:48:35 on dist-test-slave-10pc"
server_key: "ba719f152affb3663773cb207f1c6238"
server_key_iv: "3df8e66a620cee3c25edbd74cd0d5279"
server_key_version: "encryptionkey@0"
I20260809 07:48:35.642803 21950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83"
format_stamp: "Formatted at 2026-08-09 07:48:35 on dist-test-slave-10pc"
server_key: "ba719f152affb3663773cb207f1c6238"
server_key_iv: "3df8e66a620cee3c25edbd74cd0d5279"
server_key_version: "encryptionkey@0"
I20260809 07:48:35.646587 21950 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:35.649159 21965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.650434 21950 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:35.650533 21950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83"
format_stamp: "Formatted at 2026-08-09 07:48:35 on dist-test-slave-10pc"
server_key: "ba719f152affb3663773cb207f1c6238"
server_key_iv: "3df8e66a620cee3c25edbd74cd0d5279"
server_key_version: "encryptionkey@0"
I20260809 07:48:35.650625 21950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:35.676528 21950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:35.679471 21950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:35.679636 21950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:35.686807 21950 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:46821
I20260809 07:48:35.687108 22023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:46821 every 8 connection(s)
I20260809 07:48:35.687898 21950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:35.689713 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 21950
I20260809 07:48:35.689842 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:35.690171 18717 external_mini_cluster.cc:1428] Setting key 905bb53f00d5994c1d59e10a55364812
I20260809 07:48:35.691124 22024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:35.696916 22024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83: Bootstrap starting.
Aug 09 07:48:35 dist-test-slave-10pc krb5kdc[21936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:35.699276 22024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:35.700158 22024 log.cc:824] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:35.702028 22024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83: No bootstrap required, opened a new log
I20260809 07:48:35.704583 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:35.691591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.18.71.126:46821)
0809 07:48:35.692034 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:35.692049 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:35.692082 (+ 33us) server_negotiation.cc:407] Connection header received
0809 07:48:35.692820 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:35.692838 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:35.693153 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:35.693481 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:35.694323 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.695296 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.696189 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.696729 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.696916 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:35.699357 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:35.699362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:35.699396 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:35.699428 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:35.701397 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:35.701939 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:35.701944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:35.701951 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:35.702122 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:35.702481 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:35.702485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:35.702487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:35.702908 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:35.703126 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:35.703485 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 07:48:35.703782 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":141,"threads_started":1}
I20260809 07:48:35.705735 22024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46821 } }
I20260809 07:48:35.706012 22024 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:35.706066 22024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d8a2fdd5d664e2e8d8da0e1913f3b83, State: Initialized, Role: FOLLOWER
I20260809 07:48:35.706514 22024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46821 } }
I20260809 07:48:35.706684 22024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:35.706754 22024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:35.706883 22024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:35.707777 22024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46821 } }
I20260809 07:48:35.708138 22024 leader_election.cc:302] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d8a2fdd5d664e2e8d8da0e1913f3b83; no voters:
I20260809 07:48:35.708520 22024 leader_election.cc:288] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:35.708675 22030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:35.708930 22030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [term 1 LEADER]: Becoming Leader. State: Replica: 3d8a2fdd5d664e2e8d8da0e1913f3b83, State: Running, Role: LEADER
I20260809 07:48:35.709295 22030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46821 } }
I20260809 07:48:35.709867 22024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:35.711174 22032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d8a2fdd5d664e2e8d8da0e1913f3b83. Latest consensus state: current_term: 1 leader_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46821 } } }
I20260809 07:48:35.711162 22031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d8a2fdd5d664e2e8d8da0e1913f3b83" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46821 } } }
I20260809 07:48:35.711303 22032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:35.711303 22031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:35.711706 22040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:35.714426 22040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:35.719591 22040 catalog_manager.cc:1347] Generated new cluster ID: 58bc42f067594982848fe65561e4d430
I20260809 07:48:35.719659 22040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:35.753557 22040 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:35.754941 22040 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:35.764140 22040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d8a2fdd5d664e2e8d8da0e1913f3b83: Generated new TSK 0
I20260809 07:48:35.765031 22040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:35.799289 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46821
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:35.914354 22053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:35.914634 22053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:35.914707 22053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:35.918758 22053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:35.918843 22053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:35.918931 22053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:35.923808 22053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:35.924875 22053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:35.925899 22053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:35.935503 22060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:35.935457 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:35.935592 22053 server_base.cc:1034] running on GCE node
W20260809 07:48:35.935456 22062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:35.936232 22053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:35.936930 22053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:35.938124 22053 hybrid_clock.cc:648] HybridClock initialized: now 1786261715938108 us; error 33 us; skew 500 ppm
Aug 09 07:48:35 dist-test-slave-10pc krb5kdc[21936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:35.940984 22053 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:35.941905 22053 webserver.cc:460] Webserver started at http://127.18.71.65:44471/ using document root <none> and password file <none>
I20260809 07:48:35.942480 22053 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:35.942543 22053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:35.942785 22053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:35.944885 22053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "df0c96135dff4f01a67db2f13e519bfb"
format_stamp: "Formatted at 2026-08-09 07:48:35 on dist-test-slave-10pc"
server_key: "265bbe7d60413ab43c8bef198dada12e"
server_key_iv: "3e5cecf02e14636c617ce5247252c69d"
server_key_version: "encryptionkey@0"
I20260809 07:48:35.945478 22053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "df0c96135dff4f01a67db2f13e519bfb"
format_stamp: "Formatted at 2026-08-09 07:48:35 on dist-test-slave-10pc"
server_key: "265bbe7d60413ab43c8bef198dada12e"
server_key_iv: "3e5cecf02e14636c617ce5247252c69d"
server_key_version: "encryptionkey@0"
I20260809 07:48:35.949249 22053 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:35.951642 22069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.952905 22053 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:35.953038 22053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "df0c96135dff4f01a67db2f13e519bfb"
format_stamp: "Formatted at 2026-08-09 07:48:35 on dist-test-slave-10pc"
server_key: "265bbe7d60413ab43c8bef198dada12e"
server_key_iv: "3e5cecf02e14636c617ce5247252c69d"
server_key_version: "encryptionkey@0"
I20260809 07:48:35.953142 22053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:35.963657 22053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:35.966557 22053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:35.966749 22053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:35.967303 22053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:35.968103 22053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:35.968173 22053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.968245 22053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:35.968276 22053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:35.977536 22053 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:35299
I20260809 07:48:35.977567 22191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:35299 every 8 connection(s)
I20260809 07:48:35.978436 22053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:35.986081 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 22053
I20260809 07:48:35.986222 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:35.986521 18717 external_mini_cluster.cc:1428] Setting key 0c7194574a6b109e16a1c533a7878b04
Aug 09 07:48:35 dist-test-slave-10pc krb5kdc[21936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261715, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:35.992944 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:35.980283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:38061 (local address 127.18.71.126:46821)
0809 07:48:35.980433 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:35.980437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:35.981337 (+ 900us) server_negotiation.cc:407] Connection header received
0809 07:48:35.982814 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:35.982818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:35.982865 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:35.982941 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:35.984127 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.984761 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.985977 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.986224 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.986319 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:35.988981 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:35.988987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:35.988993 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:35.989028 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:35.990539 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:35.991172 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:35.991174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:35.991175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:35.991249 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:35.991648 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:35.991654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:35.991659 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:35.991908 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:35.992015 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:35.992544 (+ 529us) server_negotiation.cc:299] Negotiation successful
0809 07:48:35.992697 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 07:48:35.993644 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:35.980728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46821 (local address 127.18.71.65:38061)
0809 07:48:35.981190 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:35.981231 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:35.982607 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:35.983142 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:35.983149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:35.983511 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:35.983985 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:35.984009 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.984878 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.984880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:35.985855 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:35.985868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:35.986376 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:35.986381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:35.986550 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:35.987195 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:35.987215 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:35.988813 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:35.990654 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:35.990662 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:35.990680 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:35.991068 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:35.991339 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:35.991342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:35.991344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:35.991536 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:35.992046 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:35.992051 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:35.992275 (+ 224us) client_negotiation.cc:769] Sending connection context
0809 07:48:35.992430 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 07:48:35.992765 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260809 07:48:35.994908 22192 heartbeater.cc:347] Connected to a master server at 127.18.71.126:46821
I20260809 07:48:35.995191 22192 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:35.995679 22192 heartbeater.cc:511] Master 127.18.71.126:46821 requested a full tablet report, sending...
I20260809 07:48:35.997181 21982 ts_manager.cc:194] Registered new tserver with Master: df0c96135dff4f01a67db2f13e519bfb (127.18.71.65:35299)
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:35.998335 21982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:38061
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:36.017894 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46821
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:36.133621 22197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:36.133940 22197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:36.133988 22197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:36.137915 22197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:36.137969 22197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:36.138024 22197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:36.143154 22197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:36.144292 22197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:36.145357 22197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:36.155061 22202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:36.155069 22203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:36.155129 22197 server_base.cc:1034] running on GCE node
W20260809 07:48:36.155333 22205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:36.155776 22197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:36.156548 22197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:36.157730 22197 hybrid_clock.cc:648] HybridClock initialized: now 1786261716157668 us; error 76 us; skew 500 ppm
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[21936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:36.160559 22197 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:36.161391 22197 webserver.cc:460] Webserver started at http://127.18.71.66:40071/ using document root <none> and password file <none>
I20260809 07:48:36.161942 22197 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:36.162038 22197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:36.162295 22197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:36.164340 22197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "db4e641fb40c4260bb7c57af0ff586d2"
format_stamp: "Formatted at 2026-08-09 07:48:36 on dist-test-slave-10pc"
server_key: "4c8c00f55b9301acaacdb07c608335b9"
server_key_iv: "bbf31edacc55c8873918d4fe568ab232"
server_key_version: "encryptionkey@0"
I20260809 07:48:36.165017 22197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "db4e641fb40c4260bb7c57af0ff586d2"
format_stamp: "Formatted at 2026-08-09 07:48:36 on dist-test-slave-10pc"
server_key: "4c8c00f55b9301acaacdb07c608335b9"
server_key_iv: "bbf31edacc55c8873918d4fe568ab232"
server_key_version: "encryptionkey@0"
I20260809 07:48:36.168825 22197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:48:36.171448 22213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:36.172729 22197 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:36.172857 22197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "db4e641fb40c4260bb7c57af0ff586d2"
format_stamp: "Formatted at 2026-08-09 07:48:36 on dist-test-slave-10pc"
server_key: "4c8c00f55b9301acaacdb07c608335b9"
server_key_iv: "bbf31edacc55c8873918d4fe568ab232"
server_key_version: "encryptionkey@0"
I20260809 07:48:36.172964 22197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:36.194823 22197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:36.198241 22197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:36.198457 22197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:36.199064 22197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:36.199908 22197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:36.199985 22197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:36.200050 22197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:36.200091 22197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:36.209178 22197 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:35387
I20260809 07:48:36.209254 22333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:35387 every 8 connection(s)
I20260809 07:48:36.210208 22197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:36.215379 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 22197
I20260809 07:48:36.215523 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:36.215858 18717 external_mini_cluster.cc:1428] Setting key 66a62adf71b92b8680e79a564aa91f93
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[21936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:36.225620 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:36.212348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:52955 (local address 127.18.71.126:46821)
0809 07:48:36.212499 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:36.212503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:36.213441 (+ 938us) server_negotiation.cc:407] Connection header received
0809 07:48:36.214797 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:36.214800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:36.214846 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:36.214924 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:36.216337 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.216925 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.218082 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.218309 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.218421 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:36.221378 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:36.221384 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:36.221390 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:36.221422 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:36.223290 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:36.223816 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:36.223822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:36.223825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:36.223925 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:36.224275 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:36.224280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:36.224282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:36.224579 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:36.224688 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:36.225271 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 07:48:36.225408 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:48:36.226337 22336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:36.212816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46821 (local address 127.18.71.66:52955)
0809 07:48:36.213287 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:36.213329 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:36.214566 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:36.215099 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:36.215106 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:36.215481 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:36.216167 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:36.216183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.217055 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.217058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:36.217955 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:36.217964 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.218467 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.218471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:36.218646 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:36.219322 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:36.219348 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:36.221190 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:36.223426 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:36.223431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:36.223442 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:36.223710 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:36.224035 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:36.224037 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:36.224038 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:36.224170 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:36.224683 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:36.224688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:36.225018 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 07:48:36.225176 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 07:48:36.225426 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:48:36.227548 22334 heartbeater.cc:347] Connected to a master server at 127.18.71.126:46821
I20260809 07:48:36.227805 22334 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:36.228307 22334 heartbeater.cc:511] Master 127.18.71.126:46821 requested a full tablet report, sending...
I20260809 07:48:36.229511 21982 ts_manager.cc:194] Registered new tserver with Master: db4e641fb40c4260bb7c57af0ff586d2 (127.18.71.66:35387)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:48:36.230245 21982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:52955
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:36.247605 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46821
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:36.362361 22339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:36.362668 22339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:36.362752 22339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:36.367050 22339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:36.367143 22339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:36.367234 22339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:36.372479 22339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46821
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:36.373530 22339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:36.374569 22339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:36.384382 22345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:36.384418 22346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:36.384398 22348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:36.384749 22339 server_base.cc:1034] running on GCE node
I20260809 07:48:36.385195 22339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:36.385816 22339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:36.386998 22339 hybrid_clock.cc:648] HybridClock initialized: now 1786261716386984 us; error 25 us; skew 500 ppm
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[21936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:36.390120 22339 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:36.391021 22339 webserver.cc:460] Webserver started at http://127.18.71.67:42541/ using document root <none> and password file <none>
I20260809 07:48:36.391594 22339 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:36.391691 22339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:36.391927 22339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:36.393991 22339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "bd35a49a96974774af5d14d53c78607e"
format_stamp: "Formatted at 2026-08-09 07:48:36 on dist-test-slave-10pc"
server_key: "4e82e6204349d14c88ea708c0a6cac70"
server_key_iv: "e4d08bc4d5f23d2b74dc7b5f3c1981f8"
server_key_version: "encryptionkey@0"
I20260809 07:48:36.394577 22339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "bd35a49a96974774af5d14d53c78607e"
format_stamp: "Formatted at 2026-08-09 07:48:36 on dist-test-slave-10pc"
server_key: "4e82e6204349d14c88ea708c0a6cac70"
server_key_iv: "e4d08bc4d5f23d2b74dc7b5f3c1981f8"
server_key_version: "encryptionkey@0"
I20260809 07:48:36.398231 22339 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:36.400703 22356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:36.401962 22339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:36.402099 22339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "bd35a49a96974774af5d14d53c78607e"
format_stamp: "Formatted at 2026-08-09 07:48:36 on dist-test-slave-10pc"
server_key: "4e82e6204349d14c88ea708c0a6cac70"
server_key_iv: "e4d08bc4d5f23d2b74dc7b5f3c1981f8"
server_key_version: "encryptionkey@0"
I20260809 07:48:36.402210 22339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:36.435444 22339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:36.438748 22339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:36.438971 22339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:36.439606 22339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:36.440523 22339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:36.440598 22339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:36.440687 22339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:36.440732 22339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:36.451102 22339 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:43045
I20260809 07:48:36.451256 22474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:43045 every 8 connection(s)
I20260809 07:48:36.452126 22339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:36.454703 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 22339
I20260809 07:48:36.454807 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:36.455127 18717 external_mini_cluster.cc:1428] Setting key 64a8cc0a6963fb66a2c05aa62046865a
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[21936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261716, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:36.466280 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:36.454238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:50627 (local address 127.18.71.126:46821)
0809 07:48:36.454379 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:36.454383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:36.455320 (+ 937us) server_negotiation.cc:407] Connection header received
0809 07:48:36.457025 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:36.457029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:36.457103 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:36.457205 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:36.458534 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.459068 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.460224 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.460493 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.460609 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:36.463027 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:36.463030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:36.463032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:36.463055 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:36.464287 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:36.464981 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:36.464983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:36.464985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:36.465061 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:36.465363 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:36.465365 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:36.465366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:36.465533 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:36.465607 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:36.466066 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 07:48:36.466164 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:48:36.467129 22477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:36.454598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46821 (local address 127.18.71.67:50627)
0809 07:48:36.455169 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:36.455220 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:36.456800 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:36.457388 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:36.457398 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:36.457791 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:36.458372 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:36.458397 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.459197 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.459201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:36.460096 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:36.460106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.460648 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.460653 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:36.460839 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:36.461379 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:36.461395 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:36.462874 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:36.464412 (+ 1538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:36.464416 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:36.464428 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:36.464879 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:36.465144 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:36.465146 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:36.465148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:36.465281 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:36.465621 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:36.465626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:36.465848 (+ 222us) client_negotiation.cc:769] Sending connection context
0809 07:48:36.466023 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 07:48:36.466266 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":187,"threads_started":1}
I20260809 07:48:36.468364 22475 heartbeater.cc:347] Connected to a master server at 127.18.71.126:46821
I20260809 07:48:36.468674 22475 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:36.469266 22475 heartbeater.cc:511] Master 127.18.71.126:46821 requested a full tablet report, sending...
I20260809 07:48:36.470351 21982 ts_manager.cc:194] Registered new tserver with Master: bd35a49a96974774af5d14d53c78607e (127.18.71.67:43045)
I20260809 07:48:36.470890 21982 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:50627
I20260809 07:48:36.480931 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[21936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261716, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[21936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261716, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:36.507745 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:36.499096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.18.71.126:46821)
0809 07:48:36.499235 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:36.499238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:36.499426 (+ 188us) server_negotiation.cc:407] Connection header received
0809 07:48:36.499538 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:36.499540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:36.499594 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:36.499673 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:36.500357 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.500932 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.501776 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:36.502024 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:36.502114 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:36.504426 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:36.504429 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:36.504432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:36.504509 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:36.506115 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:36.506564 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:36.506566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:36.506567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:36.506641 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:36.506986 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:36.506988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:36.507015 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:36.507227 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:36.507330 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:36.507533 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 07:48:36.507627 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 07:48:36.508138 21982 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40686
I20260809 07:48:36.509179 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 22053
I20260809 07:48:36.523463 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 22197
I20260809 07:48:36.530174 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 22339
I20260809 07:48:36.536995 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 21950
2026-08-09T07:48:36Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2276 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[22488](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[22488](info): set up 2 sockets
Aug 09 07:48:36 dist-test-slave-10pc krb5kdc[22488](info): commencing operation
krb5kdc: starting...
W20260809 07:48:37.620640 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:37 dist-test-slave-10pc krb5kdc[22488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:37.745781 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:39413
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:39413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:37.857517 22503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:37.857832 22503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:37.857913 22503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:37.862128 22503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:37.862216 22503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:37.862254 22503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:37.862311 22503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:37.862353 22503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:37.867678 22503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:39413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:39413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:37.868867 22503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:37.869942 22503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:37.878645 22510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.878686 22512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:37.878686 22509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:37.878789 22503 server_base.cc:1034] running on GCE node
I20260809 07:48:37.879513 22503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:37.880728 22503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:37.881892 22503 hybrid_clock.cc:648] HybridClock initialized: now 1786261717881865 us; error 38 us; skew 500 ppm
Aug 09 07:48:37 dist-test-slave-10pc krb5kdc[22488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261717, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:37.884714 22503 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:37.885674 22503 webserver.cc:460] Webserver started at http://127.18.71.126:41637/ using document root <none> and password file <none>
I20260809 07:48:37.886241 22503 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:37.886336 22503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:37.886584 22503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:37.888733 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "c5e836474a8941499024a73c9d85ff23"
format_stamp: "Formatted at 2026-08-09 07:48:37 on dist-test-slave-10pc"
server_key: "b5af069755dca64138327318b87d854f"
server_key_iv: "6b055f7fd1886ed60aecb2c0601138fd"
server_key_version: "encryptionkey@0"
I20260809 07:48:37.889312 22503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "c5e836474a8941499024a73c9d85ff23"
format_stamp: "Formatted at 2026-08-09 07:48:37 on dist-test-slave-10pc"
server_key: "b5af069755dca64138327318b87d854f"
server_key_iv: "6b055f7fd1886ed60aecb2c0601138fd"
server_key_version: "encryptionkey@0"
I20260809 07:48:37.893188 22503 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:48:37.895668 22519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:37.897002 22503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:48:37.897146 22503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "c5e836474a8941499024a73c9d85ff23"
format_stamp: "Formatted at 2026-08-09 07:48:37 on dist-test-slave-10pc"
server_key: "b5af069755dca64138327318b87d854f"
server_key_iv: "6b055f7fd1886ed60aecb2c0601138fd"
server_key_version: "encryptionkey@0"
I20260809 07:48:37.897260 22503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:37.909447 22503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:37.912390 22503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:37.912635 22503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:37.919793 22503 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:39413
I20260809 07:48:37.919811 22576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:39413 every 8 connection(s)
I20260809 07:48:37.920830 22503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:37.922008 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 22503
I20260809 07:48:37.922153 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:37.922444 18717 external_mini_cluster.cc:1428] Setting key 9f852cbd7ff68c6b121859329257af65
I20260809 07:48:37.923985 22577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:37.931162 22577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23: Bootstrap starting.
Aug 09 07:48:37 dist-test-slave-10pc krb5kdc[22488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261717, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:37.934077 22577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:37.935038 22577 log.cc:824] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:37.937104 22577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23: No bootstrap required, opened a new log
I20260809 07:48:37.938127 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:37.923873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.18.71.126:39413)
0809 07:48:37.924315 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:37.924334 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:37.924377 (+ 43us) server_negotiation.cc:407] Connection header received
0809 07:48:37.925269 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:37.925293 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:37.925709 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:37.926068 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:37.926790 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:37.928320 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:37.929267 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:37.929977 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:37.930168 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:37.932945 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:37.932951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:37.932967 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:37.932999 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:37.935039 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:37.935570 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:37.935576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:37.935581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:37.935722 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:37.936028 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:37.936031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:37.936033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:37.936517 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:37.936742 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:37.937047 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 07:48:37.937384 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":160,"threads_started":1}
I20260809 07:48:37.941148 22577 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e836474a8941499024a73c9d85ff23" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39413 } }
I20260809 07:48:37.941408 22577 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:37.941490 22577 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5e836474a8941499024a73c9d85ff23, State: Initialized, Role: FOLLOWER
I20260809 07:48:37.941922 22577 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e836474a8941499024a73c9d85ff23" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39413 } }
I20260809 07:48:37.942076 22577 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:37.942164 22577 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:37.942284 22577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:37.943207 22577 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e836474a8941499024a73c9d85ff23" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39413 } }
I20260809 07:48:37.943547 22577 leader_election.cc:302] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5e836474a8941499024a73c9d85ff23; no voters:
I20260809 07:48:37.943889 22577 leader_election.cc:288] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:37.943980 22583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:37.944267 22583 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [term 1 LEADER]: Becoming Leader. State: Replica: c5e836474a8941499024a73c9d85ff23, State: Running, Role: LEADER
I20260809 07:48:37.944653 22583 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e836474a8941499024a73c9d85ff23" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39413 } }
I20260809 07:48:37.945257 22577 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:37.946620 22584 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5e836474a8941499024a73c9d85ff23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e836474a8941499024a73c9d85ff23" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39413 } } }
I20260809 07:48:37.946620 22585 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5e836474a8941499024a73c9d85ff23. Latest consensus state: current_term: 1 leader_uuid: "c5e836474a8941499024a73c9d85ff23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5e836474a8941499024a73c9d85ff23" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39413 } } }
I20260809 07:48:37.946797 22585 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:37.946797 22584 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:37.947657 22593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:37.950837 22593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:37.956385 22593 catalog_manager.cc:1347] Generated new cluster ID: dc9031ed4e7544b783d186ad1fa84f70
I20260809 07:48:37.956471 22593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:37.998868 22593 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:37.999877 22593 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:38.005136 22593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5e836474a8941499024a73c9d85ff23: Generated new TSK 0
I20260809 07:48:38.005908 22593 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:38.042127 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39413
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:38.184851 22605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:38.185129 22605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:38.185173 22605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:38.189190 22605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:38.189245 22605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:38.189338 22605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:38.194655 22605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:38.195727 22605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:38.196794 22605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:38.206655 22612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.206650 22614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.206687 22611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:38.207039 22605 server_base.cc:1034] running on GCE node
I20260809 07:48:38.207552 22605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:38.208281 22605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:38.209474 22605 hybrid_clock.cc:648] HybridClock initialized: now 1786261718209460 us; error 47 us; skew 500 ppm
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:38.212270 22605 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:38.213198 22605 webserver.cc:460] Webserver started at http://127.18.71.65:39629/ using document root <none> and password file <none>
I20260809 07:48:38.213721 22605 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:38.213781 22605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:38.213943 22605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:38.215983 22605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "ddb9199886344d1da91cc1cbc0159d4a"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "b07762cc435622d4c44279aeb746b7b2"
server_key_iv: "ab50e271f0fc7d6853f2d6a56d98a406"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.216584 22605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "ddb9199886344d1da91cc1cbc0159d4a"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "b07762cc435622d4c44279aeb746b7b2"
server_key_iv: "ab50e271f0fc7d6853f2d6a56d98a406"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.220229 22605 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:48:38.222640 22622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.223855 22605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:38.223986 22605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "ddb9199886344d1da91cc1cbc0159d4a"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "b07762cc435622d4c44279aeb746b7b2"
server_key_iv: "ab50e271f0fc7d6853f2d6a56d98a406"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.224108 22605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:38.234247 22605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:38.237246 22605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:38.237466 22605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:38.238215 22605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:38.239360 22605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:38.239429 22605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.239492 22605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:38.239530 22605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.251174 22605 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:46323
I20260809 07:48:38.251209 22747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:46323 every 8 connection(s)
I20260809 07:48:38.252096 22605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:38.260824 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 22605
I20260809 07:48:38.261003 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:38.261369 18717 external_mini_cluster.cc:1428] Setting key 9a5d48e6697c08feee6853849d6c9d98
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:38.266748 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.253950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:35601 (local address 127.18.71.126:39413)
0809 07:48:38.254120 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:38.254126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:38.255013 (+ 887us) server_negotiation.cc:407] Connection header received
0809 07:48:38.256438 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:38.256441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:38.256542 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:38.256633 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:38.257839 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.258359 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.259507 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.259743 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.259819 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.262972 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:38.262976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:38.262982 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:38.263011 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:38.264658 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.265182 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.265187 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.265189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.265279 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.265630 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.265634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.265635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.265821 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:38.265932 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:38.266431 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 07:48:38.266563 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 07:48:38.267583 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.254358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39413 (local address 127.18.71.65:35601)
0809 07:48:38.254852 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:38.254900 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:38.256238 (+ 1338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:38.256807 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:38.256815 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:38.257220 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:38.257708 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:38.257721 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.258469 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.258471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:38.259403 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:38.259412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.259874 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.259877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:38.260064 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.260740 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:38.260763 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:38.262682 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:38.264773 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:38.264779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:38.264794 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:38.265067 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:38.265376 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:38.265379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:38.265381 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:38.265525 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:38.265931 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:38.265939 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:38.266192 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 07:48:38.266389 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 07:48:38.266715 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":113,"threads_started":1}
I20260809 07:48:38.268889 22748 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39413
I20260809 07:48:38.269146 22748 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:38.269647 22748 heartbeater.cc:511] Master 127.18.71.126:39413 requested a full tablet report, sending...
I20260809 07:48:38.271380 22537 ts_manager.cc:194] Registered new tserver with Master: ddb9199886344d1da91cc1cbc0159d4a (127.18.71.65:46323)
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:38.273061 22537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:35601
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:38.291671 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39413
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:38.407424 22753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:38.407691 22753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:38.407734 22753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:38.411811 22753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:38.411875 22753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:38.411969 22753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:38.417461 22753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:38.418581 22753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:38.419634 22753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:38.429896 22761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.429878 22758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.429863 22759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:38.430238 22753 server_base.cc:1034] running on GCE node
I20260809 07:48:38.430616 22753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:38.431241 22753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:38.432430 22753 hybrid_clock.cc:648] HybridClock initialized: now 1786261718432414 us; error 29 us; skew 500 ppm
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:38.435139 22753 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:38.436076 22753 webserver.cc:460] Webserver started at http://127.18.71.66:43915/ using document root <none> and password file <none>
I20260809 07:48:38.436715 22753 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:38.436810 22753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:38.437282 22753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:38.439574 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "ea1f7043693c41c7aa79b69fa14930a4"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "0c5f68037772d3020d5af3f67e3b87a6"
server_key_iv: "9ecbd7a966a7ec959f1e353e16ef6708"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.440167 22753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "ea1f7043693c41c7aa79b69fa14930a4"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "0c5f68037772d3020d5af3f67e3b87a6"
server_key_iv: "9ecbd7a966a7ec959f1e353e16ef6708"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.443859 22753 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:48:38.446517 22768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.447890 22753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:48:38.448048 22753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "ea1f7043693c41c7aa79b69fa14930a4"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "0c5f68037772d3020d5af3f67e3b87a6"
server_key_iv: "9ecbd7a966a7ec959f1e353e16ef6708"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.448161 22753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:38.463064 22753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:38.465896 22753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:38.466105 22753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:38.466673 22753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:38.467491 22753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:38.467563 22753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.467649 22753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:38.467703 22753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.477130 22753 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:42561
I20260809 07:48:38.477152 22891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:42561 every 8 connection(s)
I20260809 07:48:38.478082 22753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:38.478968 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 22753
I20260809 07:48:38.479095 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:38.479404 18717 external_mini_cluster.cc:1428] Setting key 267542295d58f9282770d9dc5411ad8c
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:48:38.494562 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.480088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:38235 (local address 127.18.71.126:39413)
0809 07:48:38.480248 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:38.480251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:38.481292 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 07:48:38.483395 (+ 2103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:38.483401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:38.483473 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:38.483573 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:38.484849 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.485420 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.487300 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.487520 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.487602 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.490632 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:38.490636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:38.490638 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:38.490664 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:38.492301 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.492873 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.492876 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.492878 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.492965 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.493342 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.493345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.493347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.493588 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:38.493710 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:38.494240 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 07:48:38.494381 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:48:38.495519 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.480550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39413 (local address 127.18.71.66:38235)
0809 07:48:38.481112 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:38.481172 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:38.483140 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:38.483745 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:38.483752 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:38.484116 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:38.484685 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:38.484701 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.485558 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.485565 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:38.487167 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:38.487182 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.487701 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.487709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:38.487956 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.488705 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:38.488730 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:38.490436 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:38.492428 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:38.492434 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:38.492474 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:38.492757 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:38.493063 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:38.493066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:38.493068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:38.493233 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:38.493711 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:38.493719 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:38.493983 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 07:48:38.494182 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 07:48:38.494531 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
I20260809 07:48:38.496928 22892 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39413
I20260809 07:48:38.497211 22892 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:38.497766 22892 heartbeater.cc:511] Master 127.18.71.126:39413 requested a full tablet report, sending...
I20260809 07:48:38.498926 22537 ts_manager.cc:194] Registered new tserver with Master: ea1f7043693c41c7aa79b69fa14930a4 (127.18.71.66:42561)
I20260809 07:48:38.499810 22537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:38235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:38.513623 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39413
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:38.629704 22898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:38.629963 22898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:38.630012 22898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:38.633977 22898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:38.634034 22898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:38.634090 22898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:38.639281 22898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39413
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:38.640311 22898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:38.641343 22898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:38.650916 22904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.650957 22907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:38.650957 22905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:38.651506 22898 server_base.cc:1034] running on GCE node
I20260809 07:48:38.652040 22898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:38.652824 22898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:38.654055 22898 hybrid_clock.cc:648] HybridClock initialized: now 1786261718654033 us; error 30 us; skew 500 ppm
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:38.657091 22898 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:38.658074 22898 webserver.cc:460] Webserver started at http://127.18.71.67:46447/ using document root <none> and password file <none>
I20260809 07:48:38.658730 22898 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:38.658824 22898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:38.659075 22898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:38.661414 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "a211d416b56642fc9af56a302a558d69"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "427cc285c37eaed1f8004c9f3c646bb4"
server_key_iv: "aa39a0f0131f7307aef6c4f4cc0868de"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.662024 22898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "a211d416b56642fc9af56a302a558d69"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "427cc285c37eaed1f8004c9f3c646bb4"
server_key_iv: "aa39a0f0131f7307aef6c4f4cc0868de"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.666051 22898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:48:38.668587 22916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.669924 22898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:38.670082 22898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "a211d416b56642fc9af56a302a558d69"
format_stamp: "Formatted at 2026-08-09 07:48:38 on dist-test-slave-10pc"
server_key: "427cc285c37eaed1f8004c9f3c646bb4"
server_key_iv: "aa39a0f0131f7307aef6c4f4cc0868de"
server_key_version: "encryptionkey@0"
I20260809 07:48:38.670199 22898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:38.683403 22898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:38.686287 22898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:38.686486 22898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:38.687089 22898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:38.687913 22898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:38.687987 22898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.688050 22898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:38.688099 22898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:38.697595 22898 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:41029
I20260809 07:48:38.697619 23040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:41029 every 8 connection(s)
I20260809 07:48:38.698558 22898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:38.700937 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 22898
I20260809 07:48:38.701063 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:38.701342 18717 external_mini_cluster.cc:1428] Setting key 6856e8afe95484fbd22a66b5164e419e
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:38.712620 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.700425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:39641 (local address 127.18.71.126:39413)
0809 07:48:38.700644 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:38.700648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:38.701642 (+ 994us) server_negotiation.cc:407] Connection header received
0809 07:48:38.703601 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:38.703604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:38.703662 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:38.703736 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:38.705083 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.705601 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.706725 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.706926 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.707000 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.709439 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:38.709442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:38.709444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:38.709467 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:38.710728 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.711231 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.711234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.711235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.711314 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.711621 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.711624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.711625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.711796 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:38.711870 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:38.712333 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 07:48:38.712437 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260809 07:48:38.713419 23043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.700934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39413 (local address 127.18.71.67:39641)
0809 07:48:38.701480 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:38.701535 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:38.703359 (+ 1824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:38.703908 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:38.703915 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:38.704330 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:38.704936 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:38.704964 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.705708 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.705711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:38.706620 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:38.706629 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.707081 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.707083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:38.707225 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.707757 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:38.707773 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:38.709287 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:38.710829 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:38.710834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:38.710845 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:38.711107 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:38.711395 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:38.711397 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:38.711398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:38.711526 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:38.711880 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:38.711885 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:38.712100 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 07:48:38.712256 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 07:48:38.712563 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":166,"threads_started":1}
I20260809 07:48:38.714633 23041 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39413
I20260809 07:48:38.714916 23041 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:38.715550 23041 heartbeater.cc:511] Master 127.18.71.126:39413 requested a full tablet report, sending...
I20260809 07:48:38.716621 22537 ts_manager.cc:194] Registered new tserver with Master: a211d416b56642fc9af56a302a558d69 (127.18.71.67:41029)
I20260809 07:48:38.717141 22537 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:39641
I20260809 07:48:38.727104 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:38.755580 23049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.745924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.18.71.65:46323)
0809 07:48:38.746251 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:38.746270 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:38.746299 (+ 29us) server_negotiation.cc:407] Connection header received
0809 07:48:38.746458 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:38.746464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:38.746668 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:38.746824 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:38.747606 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.748436 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.749387 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.749753 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.749837 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.752369 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:38.752377 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:38.752390 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:38.752414 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:38.753996 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.754393 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.754398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.754399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.754478 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.754702 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.754704 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.754705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.754928 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:38.755076 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:38.755299 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 07:48:38.755425 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":72,"threads_started":1}
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[22488](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:38.767032 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:38.758922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.18.71.126:39413)
0809 07:48:38.759080 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:38.759083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:38.759134 (+ 51us) server_negotiation.cc:407] Connection header received
0809 07:48:38.759184 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:38.759187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:38.759238 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:38.759304 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:38.759926 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.760603 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.761332 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:38.761565 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:38.761660 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:38.764133 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:38.764137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:38.764139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:38.764164 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:38.765521 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.765907 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.765909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.765910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.765983 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:38.766219 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:38.766221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:38.766223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:38.766429 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:38.766534 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:38.766815 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 07:48:38.766910 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260809 07:48:38.767493 22537 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50366
I20260809 07:48:38.768208 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 22605
I20260809 07:48:38.776839 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 22753
I20260809 07:48:38.785485 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 22898
I20260809 07:48:38.792251 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 22503
2026-08-09T07:48:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2255 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[23055](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[23055](info): set up 2 sockets
Aug 09 07:48:38 dist-test-slave-10pc krb5kdc[23055](info): commencing operation
krb5kdc: starting...
W20260809 07:48:39.881086 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:39 dist-test-slave-10pc krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:40.007879 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42395
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:42395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:40.121515 23068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:40.121805 23068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:40.121848 23068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:40.125918 23068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:40.125977 23068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:40.125996 23068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:40.126013 23068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:40.126029 23068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:40.131309 23068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:42395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:40.132365 23068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:40.133435 23068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:40.142063 23074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.142033 23076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:40.142097 23068 server_base.cc:1034] running on GCE node
W20260809 07:48:40.142053 23073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:40.142877 23068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:40.144043 23068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:40.145205 23068 hybrid_clock.cc:648] HybridClock initialized: now 1786261720145190 us; error 29 us; skew 500 ppm
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:40.148082 23068 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:40.148922 23068 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:40.148990 23068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:40.149214 23068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:40.151399 23068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "f4eb3679276148b7b526ef06ec0bc7ed"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "decb166540f408f18ec7aa2495e19692"
server_key_iv: "3110855e6fa81be31e319ab001c8c775"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.151986 23068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "f4eb3679276148b7b526ef06ec0bc7ed"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "decb166540f408f18ec7aa2495e19692"
server_key_iv: "3110855e6fa81be31e319ab001c8c775"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.155712 23068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:48:40.158265 23085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.159682 23068 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:40.159832 23068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "f4eb3679276148b7b526ef06ec0bc7ed"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "decb166540f408f18ec7aa2495e19692"
server_key_iv: "3110855e6fa81be31e319ab001c8c775"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.159947 23068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:40.181270 23068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:40.184638 23068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:40.184856 23068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:40.192019 23068 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:42395
I20260809 07:48:40.192333 23140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:42395 every 8 connection(s)
I20260809 07:48:40.192767 23068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:40.194789 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 23068
I20260809 07:48:40.194928 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:40.195230 18717 external_mini_cluster.cc:1428] Setting key f4e13c4f6ade22dba4ed800ebfcbbcb8
I20260809 07:48:40.195932 23141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:40.202726 23141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed: Bootstrap starting.
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:40.205452 23141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:40.206292 23141 log.cc:824] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:40.208231 23141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed: No bootstrap required, opened a new log
I20260809 07:48:40.209493 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.196657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.18.71.126:42395)
0809 07:48:40.197094 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.197111 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.197146 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:48:40.197811 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.197828 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.198142 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.198412 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:40.199310 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.200290 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.201502 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.201967 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.202082 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.204831 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:40.204836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:40.204849 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:40.204875 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:40.206695 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.207139 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.207144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.207148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.207270 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.207566 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.207569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.207570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.207955 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:40.208162 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.208504 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.208785 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":151,"threads_started":1}
I20260809 07:48:40.212430 23141 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42395 } }
I20260809 07:48:40.212734 23141 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:40.212809 23141 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4eb3679276148b7b526ef06ec0bc7ed, State: Initialized, Role: FOLLOWER
I20260809 07:48:40.213311 23141 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42395 } }
I20260809 07:48:40.213459 23141 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:40.213543 23141 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:40.213644 23141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:40.214775 23141 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42395 } }
I20260809 07:48:40.215123 23141 leader_election.cc:302] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4eb3679276148b7b526ef06ec0bc7ed; no voters:
I20260809 07:48:40.215456 23141 leader_election.cc:288] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:40.215591 23146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:40.215830 23146 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [term 1 LEADER]: Becoming Leader. State: Replica: f4eb3679276148b7b526ef06ec0bc7ed, State: Running, Role: LEADER
I20260809 07:48:40.216166 23146 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42395 } }
I20260809 07:48:40.216741 23141 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:40.217985 23148 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4eb3679276148b7b526ef06ec0bc7ed. Latest consensus state: current_term: 1 leader_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42395 } } }
I20260809 07:48:40.218125 23148 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:40.218070 23147 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4eb3679276148b7b526ef06ec0bc7ed" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42395 } } }
I20260809 07:48:40.218169 23147 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:40.218523 23155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:40.221764 23155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:40.227105 23155 catalog_manager.cc:1347] Generated new cluster ID: fb701b562d7940798773a87baa1e3f7e
I20260809 07:48:40.227183 23155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:40.239137 23155 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:40.240388 23155 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:40.250598 23155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed: Generated new TSK 0
I20260809 07:48:40.251315 23155 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:40.281916 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42395
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:40.396526 23167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:40.396773 23167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:40.396848 23167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:40.400825 23167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:40.400874 23167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:40.400969 23167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:40.406065 23167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:40.407063 23167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:40.408062 23167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:40.418223 23172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.418277 23173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.418290 23175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:40.418500 23167 server_base.cc:1034] running on GCE node
I20260809 07:48:40.418919 23167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:40.419606 23167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:40.420789 23167 hybrid_clock.cc:648] HybridClock initialized: now 1786261720420765 us; error 35 us; skew 500 ppm
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:40.423533 23167 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:40.424258 23167 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:40.424322 23167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:40.424546 23167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:40.426512 23167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "49926b32014b45ab999b1cb316c7da69"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "e3cd911838dc41607dc93846a8a72d4d"
server_key_iv: "e04b94cbed7b1b7be8f3a77d4da696eb"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.427016 23167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "49926b32014b45ab999b1cb316c7da69"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "e3cd911838dc41607dc93846a8a72d4d"
server_key_iv: "e04b94cbed7b1b7be8f3a77d4da696eb"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.430745 23167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:40.433130 23181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.434301 23167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:40.434415 23167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "49926b32014b45ab999b1cb316c7da69"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "e3cd911838dc41607dc93846a8a72d4d"
server_key_iv: "e04b94cbed7b1b7be8f3a77d4da696eb"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.434513 23167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:40.446646 23167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:40.449559 23167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:40.449746 23167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:40.450315 23167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:40.451136 23167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:40.451182 23167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.451246 23167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:40.451277 23167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.460211 23167 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:41703
I20260809 07:48:40.460229 23303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:41703 every 8 connection(s)
I20260809 07:48:40.460861 23167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:40.468775 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 23167
I20260809 07:48:40.468911 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:40.469241 18717 external_mini_cluster.cc:1428] Setting key c9e7bb3212f66b4a57e3126c828d0767
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:40.475739 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.462544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:55007 (local address 127.18.71.126:42395)
0809 07:48:40.462677 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.462681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.463553 (+ 872us) server_negotiation.cc:407] Connection header received
0809 07:48:40.465262 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.465264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.465312 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.465385 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:40.466495 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.467223 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.468431 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.468829 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.468950 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.471799 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:40.471805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:40.471811 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:40.471846 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:40.473392 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.474016 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.474019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.474020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.474092 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.474488 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.474510 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.474515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.474758 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:40.474859 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.475338 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.475484 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 07:48:40.476334 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.462951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42395 (local address 127.18.71.65:55007)
0809 07:48:40.463382 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:40.463424 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:40.464995 (+ 1571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:40.465531 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:40.465538 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:40.465885 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:40.466364 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.466378 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.467403 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.467406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.468325 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.468334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.469151 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.469155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.469360 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.470047 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:40.470070 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:40.471637 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:40.473507 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:40.473516 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:40.473533 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:40.473919 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:40.474186 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:40.474192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:40.474194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:40.474380 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:40.474870 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:40.474875 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:40.475102 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 07:48:40.475256 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 07:48:40.475499 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":106,"threads_started":1}
I20260809 07:48:40.477773 23304 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42395
I20260809 07:48:40.478017 23304 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:40.478470 23304 heartbeater.cc:511] Master 127.18.71.126:42395 requested a full tablet report, sending...
I20260809 07:48:40.479947 23103 ts_manager.cc:194] Registered new tserver with Master: 49926b32014b45ab999b1cb316c7da69 (127.18.71.65:41703)
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:40.481103 23103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:55007
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:40.498997 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42395
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:40.612910 23309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:40.613160 23309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:40.613204 23309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:40.617484 23309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:40.617542 23309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:40.617647 23309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:40.622742 23309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:40.623654 23309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:40.624706 23309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:40.634260 23314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.634294 23317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.634320 23315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:40.634652 23309 server_base.cc:1034] running on GCE node
I20260809 07:48:40.635082 23309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:40.635684 23309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:40.636866 23309 hybrid_clock.cc:648] HybridClock initialized: now 1786261720636832 us; error 43 us; skew 500 ppm
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:40.644556 23309 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:40.645439 23309 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:40.645545 23309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:40.645829 23309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:40.648037 23309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "dbe98781537e4cee81aea9df153c8215"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "2fbc6ae554039c231e1da969b118fdfa"
server_key_iv: "0f37a86e2bcc9e6ad34c7289ca007bcb"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.648635 23309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "dbe98781537e4cee81aea9df153c8215"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "2fbc6ae554039c231e1da969b118fdfa"
server_key_iv: "0f37a86e2bcc9e6ad34c7289ca007bcb"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.652266 23309 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:40.654673 23323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.655917 23309 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:40.656072 23309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "dbe98781537e4cee81aea9df153c8215"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "2fbc6ae554039c231e1da969b118fdfa"
server_key_iv: "0f37a86e2bcc9e6ad34c7289ca007bcb"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.656183 23309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:40.670800 23309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:40.673754 23309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:40.673946 23309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:40.674515 23309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:40.675328 23309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:40.675398 23309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.675477 23309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:40.675522 23309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.684340 23309 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:45539
I20260809 07:48:40.684372 23444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:45539 every 8 connection(s)
I20260809 07:48:40.684932 23309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:40.685783 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 23309
I20260809 07:48:40.685925 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:40.686233 18717 external_mini_cluster.cc:1428] Setting key 059640cf7e29b609343783439b32d7d0
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:48:40.699853 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.686801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:60141 (local address 127.18.71.126:42395)
0809 07:48:40.686987 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.686992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.687934 (+ 942us) server_negotiation.cc:407] Connection header received
0809 07:48:40.689933 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.689935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.689983 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.690062 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:40.691294 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.691819 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.693083 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.693309 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.693416 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.695821 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:40.695824 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:40.695826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:40.695850 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:40.697684 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.698179 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.698184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.698185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.698270 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.698663 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.698666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.698668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.698898 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:40.699003 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.699540 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.699676 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 07:48:40.700695 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.687254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42395 (local address 127.18.71.66:60141)
0809 07:48:40.687795 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:40.687837 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:40.689717 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:40.690303 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:40.690313 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:40.690669 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:40.691140 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.691155 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.691948 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.691957 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.692964 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.692974 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.693456 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.693460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.693628 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.694233 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:40.694249 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:40.695654 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:40.697798 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:40.697804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:40.697819 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:40.698070 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:40.698368 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:40.698371 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:40.698373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:40.698523 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:40.699004 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:40.699011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:40.699278 (+ 267us) client_negotiation.cc:769] Sending connection context
0809 07:48:40.699494 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 07:48:40.699818 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":147,"threads_started":1}
I20260809 07:48:40.701817 23445 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42395
I20260809 07:48:40.702085 23445 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:40.702531 23445 heartbeater.cc:511] Master 127.18.71.126:42395 requested a full tablet report, sending...
I20260809 07:48:40.703644 23103 ts_manager.cc:194] Registered new tserver with Master: dbe98781537e4cee81aea9df153c8215 (127.18.71.66:45539)
I20260809 07:48:40.704525 23103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:60141
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:40.715845 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42395
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:40.829591 23450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:40.829856 23450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:40.829900 23450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:40.833930 23450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:40.833990 23450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:40.834084 23450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:40.839435 23450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42395
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:40.840479 23450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:40.841511 23450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:40.851014 23455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:40.851032 23456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:40.851150 23450 server_base.cc:1034] running on GCE node
W20260809 07:48:40.851019 23458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:40.851722 23450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:40.852404 23450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:40.853611 23450 hybrid_clock.cc:648] HybridClock initialized: now 1786261720853595 us; error 30 us; skew 500 ppm
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:40.856544 23450 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:40.857293 23450 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:40.857385 23450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:40.857620 23450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:40.859764 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "afceb95219c447c89308352579e098b5"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "fa8f5b6f59bbcbfbd62386362146b4d3"
server_key_iv: "93f2c22de9b96117d0c0276045a956c4"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.860379 23450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "afceb95219c447c89308352579e098b5"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "fa8f5b6f59bbcbfbd62386362146b4d3"
server_key_iv: "93f2c22de9b96117d0c0276045a956c4"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.864210 23450 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:48:40.866791 23464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.868208 23450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:40.868312 23450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "afceb95219c447c89308352579e098b5"
format_stamp: "Formatted at 2026-08-09 07:48:40 on dist-test-slave-10pc"
server_key: "fa8f5b6f59bbcbfbd62386362146b4d3"
server_key_iv: "93f2c22de9b96117d0c0276045a956c4"
server_key_version: "encryptionkey@0"
I20260809 07:48:40.868405 23450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:40.885217 23450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:40.888175 23450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:40.888329 23450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:40.888944 23450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:40.889767 23450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:40.889816 23450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.889858 23450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:40.889878 23450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:40.898415 23450 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:33969
I20260809 07:48:40.898442 23590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:33969 every 8 connection(s)
I20260809 07:48:40.898998 23450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:40.903203 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 23450
I20260809 07:48:40.903370 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:40.903666 18717 external_mini_cluster.cc:1428] Setting key d0a571457391e1d1fc09ac1c0b6c9ef9
Aug 09 07:48:40 dist-test-slave-10pc krb5kdc[23055](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261720, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:40.912693 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.900673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:40999 (local address 127.18.71.126:42395)
0809 07:48:40.900831 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.900836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.901604 (+ 768us) server_negotiation.cc:407] Connection header received
0809 07:48:40.902901 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.902904 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.902999 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.903096 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:40.904493 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.905244 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.906574 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.906791 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.906871 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.909453 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:40.909456 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:40.909458 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:40.909482 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:40.910892 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.911355 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.911357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.911359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.911431 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.911722 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.911724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.911725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.911887 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:40.911960 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.912425 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.912572 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:48:40.913527 23593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.901028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42395 (local address 127.18.71.67:40999)
0809 07:48:40.901452 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:40.901502 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:40.902703 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:40.903258 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:40.903264 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:40.903708 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:40.904305 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.904326 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.905376 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.905380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.906459 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.906469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.906990 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.906994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.907172 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.907709 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:40.907726 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:40.909301 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:40.911002 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:40.911007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:40.911019 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:40.911271 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:40.911506 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:40.911508 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:40.911510 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:40.911640 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:40.911970 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:40.912004 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:40.912223 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 07:48:40.912381 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 07:48:40.912708 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":107,"threads_started":1}
I20260809 07:48:40.914614 23591 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42395
I20260809 07:48:40.914856 23591 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:40.915320 23591 heartbeater.cc:511] Master 127.18.71.126:42395 requested a full tablet report, sending...
I20260809 07:48:40.916330 23103 ts_manager.cc:194] Registered new tserver with Master: afceb95219c447c89308352579e098b5 (127.18.71.67:33969)
I20260809 07:48:40.917006 23103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:40999
I20260809 07:48:40.917769 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:48:40.926337 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.919651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55390 (local address 127.18.71.126:42395)
0809 07:48:40.919759 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.919762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.919958 (+ 196us) server_negotiation.cc:407] Connection header received
0809 07:48:40.920043 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.920045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.920079 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.920141 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:40.920817 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.921326 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.922101 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.922307 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.922383 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.923632 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:40.923635 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:40.923637 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:40.923654 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:40.924860 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.925283 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.925286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.925287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.925369 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:40.925612 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:40.925614 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:40.925615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:40.925831 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:40.925939 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.926138 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.926228 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:48:40.929177 23102 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:40.932165 23102 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:40.946926 23603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.941248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33969 (local address 127.0.0.1:39542)
0809 07:48:40.942226 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:40.942239 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:40.942344 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:40.942777 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:40.942779 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:40.942797 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:40.942905 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.942913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.943849 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.943855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.945321 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.945346 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.946318 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.946322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.946441 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.946513 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 07:48:40.946624 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 07:48:40.946752 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":272,"thread_start_us":93,"threads_started":1}
I20260809 07:48:40.946938 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.941396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39542 (local address 127.18.71.67:33969)
0809 07:48:40.942087 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.942103 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.942361 (+ 258us) server_negotiation.cc:407] Connection header received
0809 07:48:40.942441 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.942445 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.942643 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.942799 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:40.943006 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.943726 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.945456 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.946186 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.946296 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.946438 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.946681 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.946775 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":117,"threads_started":1}
I20260809 07:48:40.947265 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.940640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.18.71.65:41703)
0809 07:48:40.941045 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.941051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.941441 (+ 390us) server_negotiation.cc:407] Connection header received
0809 07:48:40.941743 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.941749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.941974 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.942113 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:40.942664 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.943871 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.945803 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.946415 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.946492 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.946626 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.946984 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.947108 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":87,"threads_started":1}
I20260809 07:48:40.947335 23605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.940604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:45539 (local address 127.0.0.1:36594)
0809 07:48:40.941907 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:40.941920 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:40.942016 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:40.942645 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:40.942648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:40.942665 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:40.942763 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.942769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.944186 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.944189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.945651 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.945663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.946903 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.946907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.947006 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.947039 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 07:48:40.947112 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 07:48:40.947190 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1244,"mutex_wait_us":210,"thread_start_us":104,"threads_started":1}
I20260809 07:48:40.947741 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.940534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:41703 (local address 127.0.0.1:36020)
0809 07:48:40.941298 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:40.941352 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:40.941547 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:40.942366 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:40.942372 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:40.942401 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:40.942533 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.942539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.944033 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.944036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.945651 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:40.945663 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.946719 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.946722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:40.946816 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.946836 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 07:48:40.947515 (+ 679us) client_negotiation.cc:240] Negotiation successful
0809 07:48:40.947617 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":92,"thread_start_us":90,"threads_started":1}
I20260809 07:48:40.947772 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:40.941445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36594 (local address 127.18.71.66:45539)
0809 07:48:40.941862 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:40.941873 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:40.942013 (+ 140us) server_negotiation.cc:407] Connection header received
0809 07:48:40.942134 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:40.942141 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:40.942335 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:40.942482 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:40.942900 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.944051 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.945800 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:40.946585 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:40.946707 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:40.946835 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:40.947507 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 07:48:40.947611 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":115,"threads_started":1}
I20260809 07:48:40.950506 23374 tablet_service.cc:1467] Processing CreateTablet for tablet 67a1e271d9f74766bcb687566395af04 (DEFAULT_TABLE table=test-table [id=a3740a7a3f3d4fbda34086cee1b1406a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:40.950707 23236 tablet_service.cc:1467] Processing CreateTablet for tablet 67a1e271d9f74766bcb687566395af04 (DEFAULT_TABLE table=test-table [id=a3740a7a3f3d4fbda34086cee1b1406a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:40.950671 23519 tablet_service.cc:1467] Processing CreateTablet for tablet 67a1e271d9f74766bcb687566395af04 (DEFAULT_TABLE table=test-table [id=a3740a7a3f3d4fbda34086cee1b1406a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:40.951589 23374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a1e271d9f74766bcb687566395af04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:40.951704 23236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a1e271d9f74766bcb687566395af04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:40.951706 23519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67a1e271d9f74766bcb687566395af04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:40.957939 23608 tablet_bootstrap.cc:492] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Bootstrap starting.
I20260809 07:48:40.959883 23610 tablet_bootstrap.cc:492] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69: Bootstrap starting.
I20260809 07:48:40.960785 23608 tablet_bootstrap.cc:654] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:40.960813 23609 tablet_bootstrap.cc:492] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Bootstrap starting.
I20260809 07:48:40.961534 23608 log.cc:824] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:40.961979 23610 tablet_bootstrap.cc:654] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:40.962899 23609 tablet_bootstrap.cc:654] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:40.963725 23608 tablet_bootstrap.cc:492] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: No bootstrap required, opened a new log
I20260809 07:48:40.963709 23610 log.cc:824] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:40.963938 23608 ts_tablet_manager.cc:1397] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:48:40.964144 23609 log.cc:824] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:40.965744 23609 tablet_bootstrap.cc:492] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: No bootstrap required, opened a new log
I20260809 07:48:40.965734 23610 tablet_bootstrap.cc:492] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69: No bootstrap required, opened a new log
I20260809 07:48:40.965948 23609 ts_tablet_manager.cc:1397] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 07:48:40.965958 23610 ts_tablet_manager.cc:1397] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:48:40.967937 23608 raft_consensus.cc:348] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:40.968218 23608 raft_consensus.cc:374] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:40.968268 23608 raft_consensus.cc:729] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbe98781537e4cee81aea9df153c8215, State: Initialized, Role: FOLLOWER
I20260809 07:48:40.968798 23608 consensus_queue.cc:260] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:40.969216 23609 raft_consensus.cc:348] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:40.969468 23609 raft_consensus.cc:374] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:40.969506 23445 heartbeater.cc:503] Master 127.18.71.126:42395 was elected leader, sending a full tablet report...
I20260809 07:48:40.969520 23609 raft_consensus.cc:729] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afceb95219c447c89308352579e098b5, State: Initialized, Role: FOLLOWER
I20260809 07:48:40.969867 23608 ts_tablet_manager.cc:1428] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:48:40.969914 23609 consensus_queue.cc:260] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:40.970054 23610 raft_consensus.cc:348] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:40.970322 23610 raft_consensus.cc:374] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:40.970374 23610 raft_consensus.cc:729] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49926b32014b45ab999b1cb316c7da69, State: Initialized, Role: FOLLOWER
I20260809 07:48:40.970822 23610 consensus_queue.cc:260] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:40.971319 23609 ts_tablet_manager.cc:1428] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260809 07:48:40.971529 23304 heartbeater.cc:503] Master 127.18.71.126:42395 was elected leader, sending a full tablet report...
I20260809 07:48:40.971755 23610 ts_tablet_manager.cc:1428] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:48:40.972025 23591 heartbeater.cc:503] Master 127.18.71.126:42395 was elected leader, sending a full tablet report...
I20260809 07:48:41.042128 23616 raft_consensus.cc:482] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:41.042390 23616 raft_consensus.cc:504] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:41.043716 23616 leader_election.cc:288] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49926b32014b45ab999b1cb316c7da69 (127.18.71.65:41703), afceb95219c447c89308352579e098b5 (127.18.71.67:33969)
I20260809 07:48:41.047502 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.043936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:41703 (local address 127.18.71.66:40707)
0809 07:48:41.044145 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.044164 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.044256 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.044651 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.044654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.044673 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:41.044778 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:41.044782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.045577 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.045582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:41.046387 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:41.046396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.047060 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.047063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:41.047180 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:41.047205 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.047282 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.047351 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":1}
I20260809 07:48:41.047515 23604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.044142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:40707 (local address 127.18.71.65:41703)
0809 07:48:41.044284 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.044288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.044307 (+ 19us) server_negotiation.cc:407] Connection header received
0809 07:48:41.044370 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.044373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.044424 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.044542 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:41.044909 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.045454 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.046520 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.046930 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.047088 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:41.047159 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.047315 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.047386 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:48:41.048135 23619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.044264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33969 (local address 127.18.71.66:44455)
0809 07:48:41.044701 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.044713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.044835 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.045095 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.045098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.045113 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:41.045241 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:41.045246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.045952 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.045955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:41.046734 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:41.046743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.047772 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.047776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:41.047870 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:41.047891 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.047971 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.048035 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":178,"threads_started":1}
I20260809 07:48:41.048193 23606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.044410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:44455 (local address 127.18.71.67:33969)
0809 07:48:41.044578 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.044585 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.044798 (+ 213us) server_negotiation.cc:407] Connection header received
0809 07:48:41.044955 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.044962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.045004 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.045083 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:41.045347 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.045844 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.046855 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.047454 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.047623 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:41.047686 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.048009 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.048084 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:48:41.048303 23256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a1e271d9f74766bcb687566395af04" candidate_uuid: "dbe98781537e4cee81aea9df153c8215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49926b32014b45ab999b1cb316c7da69" is_pre_election: true
I20260809 07:48:41.048662 23256 raft_consensus.cc:2454] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe98781537e4cee81aea9df153c8215 in term 0.
I20260809 07:48:41.048929 23540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a1e271d9f74766bcb687566395af04" candidate_uuid: "dbe98781537e4cee81aea9df153c8215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afceb95219c447c89308352579e098b5" is_pre_election: true
I20260809 07:48:41.049212 23325 leader_election.cc:302] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49926b32014b45ab999b1cb316c7da69, dbe98781537e4cee81aea9df153c8215; no voters:
I20260809 07:48:41.049232 23540 raft_consensus.cc:2454] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dbe98781537e4cee81aea9df153c8215 in term 0.
I20260809 07:48:41.049491 23616 raft_consensus.cc:2781] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:41.049608 23616 raft_consensus.cc:482] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:41.049665 23616 raft_consensus.cc:3037] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:41.050736 23616 raft_consensus.cc:504] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:41.051077 23616 leader_election.cc:288] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [CANDIDATE]: Term 1 election: Requested vote from peers 49926b32014b45ab999b1cb316c7da69 (127.18.71.65:41703), afceb95219c447c89308352579e098b5 (127.18.71.67:33969)
I20260809 07:48:41.051403 23256 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a1e271d9f74766bcb687566395af04" candidate_uuid: "dbe98781537e4cee81aea9df153c8215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49926b32014b45ab999b1cb316c7da69"
I20260809 07:48:41.051481 23540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67a1e271d9f74766bcb687566395af04" candidate_uuid: "dbe98781537e4cee81aea9df153c8215" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afceb95219c447c89308352579e098b5"
I20260809 07:48:41.051549 23256 raft_consensus.cc:3037] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:41.051618 23540 raft_consensus.cc:3037] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:41.052650 23256 raft_consensus.cc:2454] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe98781537e4cee81aea9df153c8215 in term 1.
I20260809 07:48:41.052783 23540 raft_consensus.cc:2454] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dbe98781537e4cee81aea9df153c8215 in term 1.
I20260809 07:48:41.053104 23325 leader_election.cc:302] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: afceb95219c447c89308352579e098b5, dbe98781537e4cee81aea9df153c8215; no voters:
I20260809 07:48:41.053314 23616 raft_consensus.cc:2781] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:41.053567 23616 raft_consensus.cc:686] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 1 LEADER]: Becoming Leader. State: Replica: dbe98781537e4cee81aea9df153c8215, State: Running, Role: LEADER
I20260809 07:48:41.053907 23616 consensus_queue.cc:237] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } }
I20260809 07:48:41.057436 23102 catalog_manager.cc:5495] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 reported cstate change: term changed from 0 to 1, leader changed from <none> to dbe98781537e4cee81aea9df153c8215 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "dbe98781537e4cee81aea9df153c8215" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dbe98781537e4cee81aea9df153c8215" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 45539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 } health_report { overall_health: UNKNOWN } } }
I20260809 07:48:41.068495 23607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.065205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.18.71.66:45539)
0809 07:48:41.065318 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.065322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.065334 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:48:41.065449 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.065452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.065502 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.065587 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:41.065805 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.066319 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.067116 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:41.067313 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:41.067392 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:41.067629 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:41.068078 (+ 449us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:41.068189 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.068261 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.068333 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:48:41.074316 23540 raft_consensus.cc:1261] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 1 FOLLOWER]: Refusing update from remote peer dbe98781537e4cee81aea9df153c8215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:41.074383 23256 raft_consensus.cc:1261] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 1 FOLLOWER]: Refusing update from remote peer dbe98781537e4cee81aea9df153c8215: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:41.075057 23616 consensus_queue.cc:1035] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49926b32014b45ab999b1cb316c7da69" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 41703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:41.075214 23620 consensus_queue.cc:1035] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afceb95219c447c89308352579e098b5" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:41.084002 23624 mvcc.cc:205] Tried to move back new op lower bound from 7316528009513152512 to 7316528009440006144. Current Snapshot: MvccSnapshot[applied={T|T < 7316528009513152512}]
I20260809 07:48:41.087508 23625 mvcc.cc:205] Tried to move back new op lower bound from 7316528009513152512 to 7316528009440006144. Current Snapshot: MvccSnapshot[applied={T|T < 7316528009513152512}]
I20260809 07:48:41.093035 23102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:41.093196 23102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:41.095701 23102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4eb3679276148b7b526ef06ec0bc7ed: Sending DeleteTablet for 3 replicas of tablet 67a1e271d9f74766bcb687566395af04
I20260809 07:48:41.096647 23519 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a1e271d9f74766bcb687566395af04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:41 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:39542
I20260809 07:48:41.096836 23236 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a1e271d9f74766bcb687566395af04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:41 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:36020
I20260809 07:48:41.096871 23374 tablet_service.cc:1514] Processing DeleteTablet for tablet 67a1e271d9f74766bcb687566395af04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:41 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:36594
I20260809 07:48:41.097519 23634 tablet_replica.cc:330] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69: stopping tablet replica
I20260809 07:48:41.097641 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 23167
I20260809 07:48:41.097890 23634 raft_consensus.cc:2229] T 67a1e271d9f74766bcb687566395af04 P 49926b32014b45ab999b1cb316c7da69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:41.098357 23633 tablet_replica.cc:330] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: stopping tablet replica
I20260809 07:48:41.098728 23633 raft_consensus.cc:2229] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:41.099123 23633 raft_consensus.cc:2258] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:41.099713 23635 tablet_replica.cc:330] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: stopping tablet replica
I20260809 07:48:41.100118 23635 raft_consensus.cc:2229] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:41.101011 23635 raft_consensus.cc:2258] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:41.104136 23633 ts_tablet_manager.cc:1905] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:41.105043 23635 ts_tablet_manager.cc:1905] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:41.106974 23633 ts_tablet_manager.cc:1918] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:41.107146 23633 log.cc:1196] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/67a1e271d9f74766bcb687566395af04
I20260809 07:48:41.107547 23633 ts_tablet_manager.cc:1939] T 67a1e271d9f74766bcb687566395af04 P afceb95219c447c89308352579e098b5: Deleting consensus metadata
I20260809 07:48:41.108882 23087 catalog_manager.cc:4841] TS afceb95219c447c89308352579e098b5 (127.18.71.67:33969): tablet 67a1e271d9f74766bcb687566395af04 (table test-table [id=a3740a7a3f3d4fbda34086cee1b1406a]) successfully deleted
I20260809 07:48:41.109097 23635 ts_tablet_manager.cc:1918] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:41.109223 23635 log.cc:1196] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/67a1e271d9f74766bcb687566395af04
I20260809 07:48:41.109578 23635 ts_tablet_manager.cc:1939] T 67a1e271d9f74766bcb687566395af04 P dbe98781537e4cee81aea9df153c8215: Deleting consensus metadata
I20260809 07:48:41.110706 23089 catalog_manager.cc:4841] TS dbe98781537e4cee81aea9df153c8215 (127.18.71.66:45539): tablet 67a1e271d9f74766bcb687566395af04 (table test-table [id=a3740a7a3f3d4fbda34086cee1b1406a]) successfully deleted
W20260809 07:48:41.117658 23087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:41703 (error 108)
I20260809 07:48:41.117830 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 23309
I20260809 07:48:41.119462 23602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.119042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:41703 (local address 127.0.0.1:36028)
0809 07:48:41.119277 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.119362 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:41703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260809 07:48:41.119736 23087 catalog_manager.cc:4584] TS 49926b32014b45ab999b1cb316c7da69 (127.18.71.65:41703): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67a1e271d9f74766bcb687566395af04: Network error: Client connection negotiation failed: client connection to 127.18.71.65:41703: connect: Connection refused (error 111)
I20260809 07:48:41.126885 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 23450
I20260809 07:48:41.134300 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 23068
2026-08-09T07:48:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2343 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T07:48:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:41Z Disabled control of system clock
I20260809 07:48:41.168248 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42021
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:42021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 07:48:41.281260 23640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:41.281513 23640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:41.281550 23640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:41.285799 23640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:41.285858 23640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:41.285873 23640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:41.285912 23640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:41.285943 23640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:41.290719 23640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:42021
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:41.291807 23640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:41.293332 23640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:41.301797 23645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.301759 23646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:41.301975 23640 server_base.cc:1034] running on GCE node
W20260809 07:48:41.301826 23648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:41.302955 23640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:41.303977 23640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:41.305140 23640 hybrid_clock.cc:648] HybridClock initialized: now 1786261721305111 us; error 40 us; skew 500 ppm
I20260809 07:48:41.307009 23640 webserver.cc:460] Webserver started at http://127.18.71.126:39407/ using document root <none> and password file <none>
I20260809 07:48:41.307596 23640 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:41.307662 23640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:41.307901 23640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:41.310024 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "6193253d86e6458393fa87ca3be2c9e2"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "9fe9ef193d95fb06df61ba5a222b7391"
server_key_iv: "c5186635f477a5c6e965c57cae428186"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.310599 23640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "6193253d86e6458393fa87ca3be2c9e2"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "9fe9ef193d95fb06df61ba5a222b7391"
server_key_iv: "c5186635f477a5c6e965c57cae428186"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.314397 23640 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:48:41.316818 23654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.318037 23640 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:41.318186 23640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "6193253d86e6458393fa87ca3be2c9e2"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "9fe9ef193d95fb06df61ba5a222b7391"
server_key_iv: "c5186635f477a5c6e965c57cae428186"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.318301 23640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:41.322974 23640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:41.323624 23640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:41.323809 23640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:41.330658 23640 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:42021
I20260809 07:48:41.330668 23710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:42021 every 8 connection(s)
I20260809 07:48:41.331595 23640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:41.333900 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 23640
I20260809 07:48:41.334035 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:41.334311 18717 external_mini_cluster.cc:1428] Setting key b5c3c53317bfd12cf54b9070080159bb
I20260809 07:48:41.335192 23711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:41.340032 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.335820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55986 (local address 127.18.71.126:42021)
0809 07:48:41.336545 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.336560 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.336597 (+ 37us) server_negotiation.cc:407] Connection header received
0809 07:48:41.337274 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.337291 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.337638 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.337965 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.338786 (+ 821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.338791 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.338802 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.338813 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.338870 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.338885 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.339033 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.339107 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.339353 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":154,"threads_started":1}
I20260809 07:48:41.341822 23711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2: Bootstrap starting.
I20260809 07:48:41.344122 23711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:41.345006 23711 log.cc:824] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:41.346879 23711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2: No bootstrap required, opened a new log
I20260809 07:48:41.350292 23711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6193253d86e6458393fa87ca3be2c9e2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42021 } }
I20260809 07:48:41.350551 23711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:41.350601 23711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6193253d86e6458393fa87ca3be2c9e2, State: Initialized, Role: FOLLOWER
I20260809 07:48:41.350955 23711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6193253d86e6458393fa87ca3be2c9e2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42021 } }
I20260809 07:48:41.351065 23711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:41.351109 23711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:41.351177 23711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:41.352221 23711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6193253d86e6458393fa87ca3be2c9e2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42021 } }
I20260809 07:48:41.352618 23711 leader_election.cc:302] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6193253d86e6458393fa87ca3be2c9e2; no voters:
I20260809 07:48:41.352893 23711 leader_election.cc:288] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:41.353031 23716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:41.353264 23716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [term 1 LEADER]: Becoming Leader. State: Replica: 6193253d86e6458393fa87ca3be2c9e2, State: Running, Role: LEADER
I20260809 07:48:41.353567 23716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6193253d86e6458393fa87ca3be2c9e2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42021 } }
I20260809 07:48:41.354081 23711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:41.355185 23718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6193253d86e6458393fa87ca3be2c9e2. Latest consensus state: current_term: 1 leader_uuid: "6193253d86e6458393fa87ca3be2c9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6193253d86e6458393fa87ca3be2c9e2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42021 } } }
I20260809 07:48:41.355314 23718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:41.355598 23717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6193253d86e6458393fa87ca3be2c9e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6193253d86e6458393fa87ca3be2c9e2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42021 } } }
I20260809 07:48:41.355708 23717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:41.355782 23725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:41.358920 23725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:41.364517 23725 catalog_manager.cc:1347] Generated new cluster ID: a3afaff593c64b5c9fe585e0177513e3
I20260809 07:48:41.364596 23725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:41.392161 23725 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:41.392575 23725 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:41.398898 23725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2: Generated new TSK 0
I20260809 07:48:41.399694 23725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:48:41.404034 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42021
--builtin_ntp_servers=127.18.71.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 07:48:41.519836 23737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:41.520145 23737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:41.520213 23737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:41.524358 23737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:41.524415 23737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:41.524484 23737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:41.529192 23737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:41.530283 23737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:41.531690 23737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:41.541368 23745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.541395 23747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.541399 23744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:41.541688 23737 server_base.cc:1034] running on GCE node
I20260809 07:48:41.542111 23737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:41.542752 23737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:41.543932 23737 hybrid_clock.cc:648] HybridClock initialized: now 1786261721543890 us; error 52 us; skew 500 ppm
I20260809 07:48:41.545888 23737 webserver.cc:460] Webserver started at http://127.18.71.65:42129/ using document root <none> and password file <none>
I20260809 07:48:41.546466 23737 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:41.546561 23737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:41.546788 23737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:41.548895 23737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "4e3f740c34a54021b8e7b37d0485bb7d"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "eb0a0b68201317f358a9aa1120b5a123"
server_key_iv: "ba98ced80366e5c9019f3d4b5a2090c0"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.549471 23737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "4e3f740c34a54021b8e7b37d0485bb7d"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "eb0a0b68201317f358a9aa1120b5a123"
server_key_iv: "ba98ced80366e5c9019f3d4b5a2090c0"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.553213 23737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:41.555945 23753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.557222 23737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:41.557353 23737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "4e3f740c34a54021b8e7b37d0485bb7d"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "eb0a0b68201317f358a9aa1120b5a123"
server_key_iv: "ba98ced80366e5c9019f3d4b5a2090c0"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.557461 23737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:41.562155 23737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:41.562747 23737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:41.562938 23737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:41.563472 23737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:41.564376 23737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:41.564445 23737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.564536 23737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:41.564582 23737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.573323 23737 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:32967
I20260809 07:48:41.573377 23875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:32967 every 8 connection(s)
I20260809 07:48:41.574330 23737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:41.581116 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.576445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:58959 (local address 127.18.71.126:42021)
0809 07:48:41.576626 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.576630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.577869 (+ 1239us) server_negotiation.cc:407] Connection header received
0809 07:48:41.579344 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.579348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.579394 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.579497 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.580633 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.580636 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.580639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.580652 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.580678 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.580687 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.580777 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.580932 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.580979 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:48:41.581228 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 23737
I20260809 07:48:41.581359 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:41.581651 18717 external_mini_cluster.cc:1428] Setting key c12021420a393dd97283803b0a9f8b09
I20260809 07:48:41.582094 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.577017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42021 (local address 127.18.71.65:58959)
0809 07:48:41.577718 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.577759 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.579134 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.579673 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.579680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.579770 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:41.580329 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:41.580331 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:41.580371 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:41.580441 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:41.580774 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:41.580785 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:41.580793 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.580986 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.581247 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":135,"threads_started":1}
I20260809 07:48:41.583477 23876 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42021
I20260809 07:48:41.583738 23876 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:41.583930 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42021
--builtin_ntp_servers=127.18.71.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 07:48:41.584265 23876 heartbeater.cc:511] Master 127.18.71.126:42021 requested a full tablet report, sending...
I20260809 07:48:41.586328 23672 ts_manager.cc:194] Registered new tserver with Master: 4e3f740c34a54021b8e7b37d0485bb7d (127.18.71.65:32967)
W20260809 07:48:41.702171 23879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:41.702404 23879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:41.702441 23879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:41.706307 23879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:41.706354 23879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:41.706406 23879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:41.711066 23879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:41.712028 23879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:41.713371 23879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:41.722807 23885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.722833 23888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.722857 23886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:41.723044 23879 server_base.cc:1034] running on GCE node
I20260809 07:48:41.723549 23879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:41.724186 23879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:41.725361 23879 hybrid_clock.cc:648] HybridClock initialized: now 1786261721725308 us; error 65 us; skew 500 ppm
I20260809 07:48:41.727164 23879 webserver.cc:460] Webserver started at http://127.18.71.66:38357/ using document root <none> and password file <none>
I20260809 07:48:41.727739 23879 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:41.727828 23879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:41.728056 23879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:41.730155 23879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "6a5a0fb541dad758680de8eaddaf5a67"
server_key_iv: "3729b4e286348fe8e64a52238368c3a1"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.730710 23879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "6a5a0fb541dad758680de8eaddaf5a67"
server_key_iv: "3729b4e286348fe8e64a52238368c3a1"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.734373 23879 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:48:41.736773 23894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.738021 23879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:48:41.738157 23879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "6a5a0fb541dad758680de8eaddaf5a67"
server_key_iv: "3729b4e286348fe8e64a52238368c3a1"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.738267 23879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:41.742806 23879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:41.743394 23879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:41.743566 23879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:41.744079 23879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:41.745031 23879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:41.745098 23879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.745186 23879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:41.745231 23879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.754135 23879 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:41027
I20260809 07:48:41.754212 24016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:41027 every 8 connection(s)
I20260809 07:48:41.755101 23879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:41.761690 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.757112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:52539 (local address 127.18.71.126:42021)
0809 07:48:41.757282 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.757287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.758358 (+ 1071us) server_negotiation.cc:407] Connection header received
0809 07:48:41.759826 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.759828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.759864 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.759938 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.761250 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.761253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.761255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.761265 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.761283 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.761290 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.761389 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.761545 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.761581 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:48:41.761814 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 23879
I20260809 07:48:41.761976 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:41.762391 18717 external_mini_cluster.cc:1428] Setting key 4070259f6bf0fd724227c2c0f785704d
I20260809 07:48:41.762969 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.757483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42021 (local address 127.18.71.66:52539)
0809 07:48:41.758201 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.758244 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.759626 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.760131 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.760139 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.760233 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:41.760954 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:41.760956 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:41.761005 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:41.761081 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:41.761375 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:41.761389 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:41.761398 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.761594 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.761905 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
I20260809 07:48:41.764578 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42021
--builtin_ntp_servers=127.18.71.84:45747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 07:48:41.764573 24017 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42021
I20260809 07:48:41.764822 24017 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:41.765224 24017 heartbeater.cc:511] Master 127.18.71.126:42021 requested a full tablet report, sending...
I20260809 07:48:41.766201 23672 ts_manager.cc:194] Registered new tserver with Master: a32a3da0fafe4c1cbf6fd5cc1fd502e7 (127.18.71.66:41027)
W20260809 07:48:41.880165 24020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:41.880419 24020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:41.880518 24020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:41.884521 24020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:41.884573 24020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:41.884655 24020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:41.889875 24020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:41.890913 24020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:41.892297 24020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:41.901921 24026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.901947 24030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:41.901917 24027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:41.902205 24020 server_base.cc:1034] running on GCE node
I20260809 07:48:41.902664 24020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:41.903319 24020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:41.904544 24020 hybrid_clock.cc:648] HybridClock initialized: now 1786261721904521 us; error 32 us; skew 500 ppm
I20260809 07:48:41.906526 24020 webserver.cc:460] Webserver started at http://127.18.71.67:40187/ using document root <none> and password file <none>
I20260809 07:48:41.907122 24020 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:41.907217 24020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:41.907433 24020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:41.909559 24020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "032a89d8734d4ef5b2a9a67b70cfbae3"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "0a662253cb465987e5c5621c19464134"
server_key_iv: "0147206f5dea15e41b543772a75f53e6"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.910151 24020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "032a89d8734d4ef5b2a9a67b70cfbae3"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "0a662253cb465987e5c5621c19464134"
server_key_iv: "0147206f5dea15e41b543772a75f53e6"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.913813 24020 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:41.916580 24036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.917819 24020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:41.917963 24020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "032a89d8734d4ef5b2a9a67b70cfbae3"
format_stamp: "Formatted at 2026-08-09 07:48:41 on dist-test-slave-10pc"
server_key: "0a662253cb465987e5c5621c19464134"
server_key_iv: "0147206f5dea15e41b543772a75f53e6"
server_key_version: "encryptionkey@0"
I20260809 07:48:41.918116 24020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:41.922823 24020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:41.923448 24020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:41.923633 24020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:41.924189 24020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:41.925161 24020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:41.925230 24020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.925297 24020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:41.925344 24020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:41.933887 24020 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:36045
I20260809 07:48:41.933907 24158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:36045 every 8 connection(s)
I20260809 07:48:41.934796 24020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:41.940861 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.936648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:58639 (local address 127.18.71.126:42021)
0809 07:48:41.936795 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.936800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.937954 (+ 1154us) server_negotiation.cc:407] Connection header received
0809 07:48:41.939226 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.939229 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.939268 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.939346 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.940398 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.940401 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.940404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.940415 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.940435 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.940443 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.940591 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.940714 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.940752 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:48:41.941745 24161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.937125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42021 (local address 127.18.71.67:58639)
0809 07:48:41.937775 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.937816 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.939013 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.939502 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.939508 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.939600 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:41.940146 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:41.940147 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:41.940187 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:41.940255 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:41.940587 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:41.940594 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:41.940599 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.940737 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.940921 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":110,"threads_started":1}
I20260809 07:48:41.942282 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24020
I20260809 07:48:41.942414 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:41.942746 18717 external_mini_cluster.cc:1428] Setting key 204c0879e16c73adcfef4836336c6b1e
I20260809 07:48:41.943007 24159 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42021
I20260809 07:48:41.943264 24159 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:41.943756 24159 heartbeater.cc:511] Master 127.18.71.126:42021 requested a full tablet report, sending...
I20260809 07:48:41.944811 23672 ts_manager.cc:194] Registered new tserver with Master: 032a89d8734d4ef5b2a9a67b70cfbae3 (127.18.71.67:36045)
I20260809 07:48:41.956660 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:48:41.960091 23714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.958582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.18.71.126:42021)
0809 07:48:41.958733 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.958738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.958970 (+ 232us) server_negotiation.cc:407] Connection header received
0809 07:48:41.959058 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.959060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.959095 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.959162 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.959752 (+ 590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.959754 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.959756 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.959765 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.959783 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.959790 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.959854 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.959893 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.959923 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 07:48:41.962977 23672 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56002:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:41.966082 23672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:41.977322 24170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.974377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:32967 (local address 127.0.0.1:58124)
0809 07:48:41.975274 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.975288 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.975355 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.975993 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.975997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.976014 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:41.976663 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:41.976667 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:41.976681 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:41.976709 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:41.977053 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:41.977058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:41.977063 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.977136 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.977158 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":119,"threads_started":1}
I20260809 07:48:41.977324 24171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.974192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:36045 (local address 127.0.0.1:33994)
0809 07:48:41.974762 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.974815 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.974916 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.975697 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.975703 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.975779 (+ 76us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:41.976374 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:41.976379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:41.976402 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:41.976509 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:41.976944 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:41.976955 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:41.976968 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.977081 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.977155 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":79,"threads_started":1}
I20260809 07:48:41.977676 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.974914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58124 (local address 127.18.71.65:32967)
0809 07:48:41.975604 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.975617 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.975643 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:48:41.975696 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.975701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.975859 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.976015 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.976840 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.976846 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.976858 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.976869 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.976911 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.976958 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.977073 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.977158 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.977243 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":143,"threads_started":1}
I20260809 07:48:41.978207 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.974379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33994 (local address 127.18.71.67:36045)
0809 07:48:41.975189 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.975204 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.975231 (+ 27us) server_negotiation.cc:407] Connection header received
0809 07:48:41.975303 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.975311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.975524 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.976064 (+ 540us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.976719 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.976725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.976738 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.976749 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.976791 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.976823 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.977905 (+ 1082us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.977955 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.978042 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":150,"threads_started":1}
I20260809 07:48:41.978591 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.974908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:41027 (local address 127.0.0.1:59358)
0809 07:48:41.975697 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:41.975712 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:41.975791 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:41.976755 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:41.976758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:41.976772 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:41.977541 (+ 769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:41.977543 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:41.977553 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:41.977580 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:41.978340 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:41.978345 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:41.978349 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 07:48:41.978435 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 07:48:41.978470 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":96,"threads_started":1}
I20260809 07:48:41.978703 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:41.975301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59358 (local address 127.18.71.66:41027)
0809 07:48:41.975740 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:41.975754 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:41.975779 (+ 25us) server_negotiation.cc:407] Connection header received
0809 07:48:41.975893 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:41.975898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:41.976617 (+ 719us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:41.976742 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:41.978024 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:41.978029 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:41.978039 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:41.978049 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:41.978083 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:41.978103 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:41.978211 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:41.978484 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 07:48:41.978566 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":104,"threads_started":1}
I20260809 07:48:41.980942 23804 tablet_service.cc:1467] Processing CreateTablet for tablet 2b2ceadf004c475dbb36fd9ae25c7bda (DEFAULT_TABLE table=test-table [id=5680b816e3af448c85c9b73c4fdc963f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:41.980942 24087 tablet_service.cc:1467] Processing CreateTablet for tablet 2b2ceadf004c475dbb36fd9ae25c7bda (DEFAULT_TABLE table=test-table [id=5680b816e3af448c85c9b73c4fdc963f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:41.981462 23945 tablet_service.cc:1467] Processing CreateTablet for tablet 2b2ceadf004c475dbb36fd9ae25c7bda (DEFAULT_TABLE table=test-table [id=5680b816e3af448c85c9b73c4fdc963f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:41.982110 23804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b2ceadf004c475dbb36fd9ae25c7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:41.982110 24087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b2ceadf004c475dbb36fd9ae25c7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:41.982437 23945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b2ceadf004c475dbb36fd9ae25c7bda. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:41.988765 24176 tablet_bootstrap.cc:492] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Bootstrap starting.
I20260809 07:48:41.989485 24178 tablet_bootstrap.cc:492] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: Bootstrap starting.
I20260809 07:48:41.991894 24178 tablet_bootstrap.cc:654] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:41.992364 24176 tablet_bootstrap.cc:654] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:41.992929 24178 log.cc:824] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:41.993214 24176 log.cc:824] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:41.993686 24177 tablet_bootstrap.cc:492] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Bootstrap starting.
I20260809 07:48:41.994982 24176 tablet_bootstrap.cc:492] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: No bootstrap required, opened a new log
I20260809 07:48:41.995141 24178 tablet_bootstrap.cc:492] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: No bootstrap required, opened a new log
I20260809 07:48:41.995201 24176 ts_tablet_manager.cc:1397] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 07:48:41.995328 24178 ts_tablet_manager.cc:1397] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:48:41.995805 24177 tablet_bootstrap.cc:654] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:41.996622 24177 log.cc:824] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:41.998674 24177 tablet_bootstrap.cc:492] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: No bootstrap required, opened a new log
I20260809 07:48:41.998857 24177 ts_tablet_manager.cc:1397] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 07:48:41.999481 24176 raft_consensus.cc:348] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:41.999737 24176 raft_consensus.cc:374] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:41.999790 24176 raft_consensus.cc:729] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32a3da0fafe4c1cbf6fd5cc1fd502e7, State: Initialized, Role: FOLLOWER
I20260809 07:48:41.999858 24178 raft_consensus.cc:348] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.000129 24178 raft_consensus.cc:374] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:42.000186 24178 raft_consensus.cc:729] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3f740c34a54021b8e7b37d0485bb7d, State: Initialized, Role: FOLLOWER
I20260809 07:48:42.000290 24176 consensus_queue.cc:260] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.000651 24178 consensus_queue.cc:260] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.001055 24017 heartbeater.cc:503] Master 127.18.71.126:42021 was elected leader, sending a full tablet report...
I20260809 07:48:42.001364 24176 ts_tablet_manager.cc:1428] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:48:42.001693 24178 ts_tablet_manager.cc:1428] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:48:42.001878 23876 heartbeater.cc:503] Master 127.18.71.126:42021 was elected leader, sending a full tablet report...
I20260809 07:48:42.003278 24177 raft_consensus.cc:348] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.003548 24177 raft_consensus.cc:374] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:42.003607 24177 raft_consensus.cc:729] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 032a89d8734d4ef5b2a9a67b70cfbae3, State: Initialized, Role: FOLLOWER
I20260809 07:48:42.003988 24177 consensus_queue.cc:260] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.004758 24159 heartbeater.cc:503] Master 127.18.71.126:42021 was elected leader, sending a full tablet report...
I20260809 07:48:42.004968 24177 ts_tablet_manager.cc:1428] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
W20260809 07:48:42.006127 24018 tablet.cc:2369] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:42.039484 24183 raft_consensus.cc:482] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:42.039727 24183 raft_consensus.cc:504] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.041021 24183 leader_election.cc:288] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 032a89d8734d4ef5b2a9a67b70cfbae3 (127.18.71.67:36045), 4e3f740c34a54021b8e7b37d0485bb7d (127.18.71.65:32967)
I20260809 07:48:42.043136 24172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.041339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:44965 (local address 127.18.71.67:36045)
0809 07:48:42.041474 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:42.041479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:42.041606 (+ 127us) server_negotiation.cc:407] Connection header received
0809 07:48:42.041740 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:42.041743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:42.041805 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.041898 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:42.042745 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:42.042748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:42.042751 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:42.042764 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:42.042786 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:42.042795 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:42.042888 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:42.042966 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 07:48:42.043009 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:48:42.043144 24174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.041455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:51399 (local address 127.18.71.65:32967)
0809 07:48:42.041650 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:42.041654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:42.041711 (+ 57us) server_negotiation.cc:407] Connection header received
0809 07:48:42.041825 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:42.041830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:42.041873 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.041953 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:42.042746 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:42.042749 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:42.042751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:42.042764 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:42.042785 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:42.042796 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:42.042888 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:42.042966 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 07:48:42.043039 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 07:48:42.043164 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.041217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:36045 (local address 127.18.71.66:44965)
0809 07:48:42.041503 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:42.041521 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:42.041628 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:42.041939 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:42.041944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:42.041959 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:42.042565 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:42.042568 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:42.042582 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:42.042611 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:42.042885 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:42.042888 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:42.042891 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 07:48:42.042969 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 07:48:42.043020 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212}
I20260809 07:48:42.043191 24186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.041253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:32967 (local address 127.18.71.66:51399)
0809 07:48:42.041641 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:42.041656 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:42.041733 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:42.041969 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:42.041972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:42.041983 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:42.042586 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:48:42.042588 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:42.042595 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:48:42.042611 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:42.042885 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:42.042888 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:42.042891 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 07:48:42.042969 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 07:48:42.043031 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":18,"thread_start_us":139,"threads_started":1}
I20260809 07:48:42.043857 23825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b2ceadf004c475dbb36fd9ae25c7bda" candidate_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" is_pre_election: true
I20260809 07:48:42.043859 24109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b2ceadf004c475dbb36fd9ae25c7bda" candidate_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" is_pre_election: true
I20260809 07:48:42.044154 24109 raft_consensus.cc:2454] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32a3da0fafe4c1cbf6fd5cc1fd502e7 in term 0.
I20260809 07:48:42.044168 23825 raft_consensus.cc:2454] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32a3da0fafe4c1cbf6fd5cc1fd502e7 in term 0.
I20260809 07:48:42.044677 23896 leader_election.cc:302] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 032a89d8734d4ef5b2a9a67b70cfbae3, 4e3f740c34a54021b8e7b37d0485bb7d, a32a3da0fafe4c1cbf6fd5cc1fd502e7; no voters:
I20260809 07:48:42.044939 24183 raft_consensus.cc:2781] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:42.045028 24183 raft_consensus.cc:482] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:42.045079 24183 raft_consensus.cc:3037] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:42.046132 24183 raft_consensus.cc:504] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.046487 24183 leader_election.cc:288] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [CANDIDATE]: Term 1 election: Requested vote from peers 032a89d8734d4ef5b2a9a67b70cfbae3 (127.18.71.67:36045), 4e3f740c34a54021b8e7b37d0485bb7d (127.18.71.65:32967)
I20260809 07:48:42.046787 24109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b2ceadf004c475dbb36fd9ae25c7bda" candidate_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3"
I20260809 07:48:42.046824 23825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b2ceadf004c475dbb36fd9ae25c7bda" candidate_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e3f740c34a54021b8e7b37d0485bb7d"
I20260809 07:48:42.046943 24109 raft_consensus.cc:3037] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:42.046953 23825 raft_consensus.cc:3037] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:42.047977 24109 raft_consensus.cc:2454] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32a3da0fafe4c1cbf6fd5cc1fd502e7 in term 1.
I20260809 07:48:42.047976 23825 raft_consensus.cc:2454] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32a3da0fafe4c1cbf6fd5cc1fd502e7 in term 1.
I20260809 07:48:42.048252 23896 leader_election.cc:302] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 032a89d8734d4ef5b2a9a67b70cfbae3, a32a3da0fafe4c1cbf6fd5cc1fd502e7; no voters:
I20260809 07:48:42.048435 24183 raft_consensus.cc:2781] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:42.048712 24183 raft_consensus.cc:686] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 1 LEADER]: Becoming Leader. State: Replica: a32a3da0fafe4c1cbf6fd5cc1fd502e7, State: Running, Role: LEADER
I20260809 07:48:42.049077 24183 consensus_queue.cc:237] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } }
I20260809 07:48:42.052441 23672 catalog_manager.cc:5495] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to a32a3da0fafe4c1cbf6fd5cc1fd502e7 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a32a3da0fafe4c1cbf6fd5cc1fd502e7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41027 } health_report { overall_health: HEALTHY } } }
W20260809 07:48:42.075699 23877 tablet.cc:2369] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:42.075891 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.074335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59366 (local address 127.18.71.66:41027)
0809 07:48:42.074487 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:42.074492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:42.074509 (+ 17us) server_negotiation.cc:407] Connection header received
0809 07:48:42.074643 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:42.074645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:42.074705 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.074796 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:42.075450 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:42.075454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:42.075457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:42.075471 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:42.075502 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:42.075512 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:42.075604 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:42.075720 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 07:48:42.075767 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:48:42.081712 23825 raft_consensus.cc:1261] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 1 FOLLOWER]: Refusing update from remote peer a32a3da0fafe4c1cbf6fd5cc1fd502e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:42.081713 24109 raft_consensus.cc:1261] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 1 FOLLOWER]: Refusing update from remote peer a32a3da0fafe4c1cbf6fd5cc1fd502e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:42.082438 24187 consensus_queue.cc:1035] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e3f740c34a54021b8e7b37d0485bb7d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:42.082664 24183 consensus_queue.cc:1035] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "032a89d8734d4ef5b2a9a67b70cfbae3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 36045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:42.091320 24192 mvcc.cc:205] Tried to move back new op lower bound from 7316528013639520256 to 7316528013515694080. Current Snapshot: MvccSnapshot[applied={T|T < 7316528013639520256}]
I20260809 07:48:42.091487 24191 mvcc.cc:205] Tried to move back new op lower bound from 7316528013639520256 to 7316528013515694080. Current Snapshot: MvccSnapshot[applied={T|T < 7316528013639520256}]
I20260809 07:48:42.099061 23670 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:42.099227 23670 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56002:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:42.101591 23670 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6193253d86e6458393fa87ca3be2c9e2: Sending DeleteTablet for 3 replicas of tablet 2b2ceadf004c475dbb36fd9ae25c7bda
I20260809 07:48:42.102319 24087 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b2ceadf004c475dbb36fd9ae25c7bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:42 UTC) from {username='slave'} at 127.0.0.1:33994
I20260809 07:48:42.102500 23804 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b2ceadf004c475dbb36fd9ae25c7bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:42 UTC) from {username='slave'} at 127.0.0.1:58124
I20260809 07:48:42.102577 23945 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b2ceadf004c475dbb36fd9ae25c7bda with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:42 UTC) from {username='slave'} at 127.0.0.1:59358
I20260809 07:48:42.102900 24200 tablet_replica.cc:330] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: stopping tablet replica
I20260809 07:48:42.103086 24201 tablet_replica.cc:330] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d: stopping tablet replica
I20260809 07:48:42.103751 24201 raft_consensus.cc:2229] T 2b2ceadf004c475dbb36fd9ae25c7bda P 4e3f740c34a54021b8e7b37d0485bb7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:42.103751 24200 raft_consensus.cc:2229] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:42.103752 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 23737
I20260809 07:48:42.105413 24202 tablet_replica.cc:330] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: stopping tablet replica
I20260809 07:48:42.105543 24200 raft_consensus.cc:2258] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:42.105715 24202 raft_consensus.cc:2229] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:42.106105 24202 raft_consensus.cc:2258] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:42.107396 24200 ts_tablet_manager.cc:1905] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:42.108954 24202 ts_tablet_manager.cc:1905] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:42.111110 24200 ts_tablet_manager.cc:1918] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:42.111282 24200 log.cc:1196] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/2b2ceadf004c475dbb36fd9ae25c7bda
I20260809 07:48:42.111665 24200 ts_tablet_manager.cc:1939] T 2b2ceadf004c475dbb36fd9ae25c7bda P 032a89d8734d4ef5b2a9a67b70cfbae3: Deleting consensus metadata
I20260809 07:48:42.111794 24202 ts_tablet_manager.cc:1918] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:42.111910 24202 log.cc:1196] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/2b2ceadf004c475dbb36fd9ae25c7bda
I20260809 07:48:42.112286 24202 ts_tablet_manager.cc:1939] T 2b2ceadf004c475dbb36fd9ae25c7bda P a32a3da0fafe4c1cbf6fd5cc1fd502e7: Deleting consensus metadata
I20260809 07:48:42.113268 23656 catalog_manager.cc:4841] TS 032a89d8734d4ef5b2a9a67b70cfbae3 (127.18.71.67:36045): tablet 2b2ceadf004c475dbb36fd9ae25c7bda (table test-table [id=5680b816e3af448c85c9b73c4fdc963f]) successfully deleted
I20260809 07:48:42.113610 23659 catalog_manager.cc:4841] TS a32a3da0fafe4c1cbf6fd5cc1fd502e7 (127.18.71.66:41027): tablet 2b2ceadf004c475dbb36fd9ae25c7bda (table test-table [id=5680b816e3af448c85c9b73c4fdc963f]) successfully deleted
W20260809 07:48:42.114904 23659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:32967 (error 108)
I20260809 07:48:42.115020 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 23879
I20260809 07:48:42.117594 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.117272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:32967 (local address 127.0.0.1:58126)
0809 07:48:42.117451 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:42.117522 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:32967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260809 07:48:42.117796 23659 catalog_manager.cc:4584] TS 4e3f740c34a54021b8e7b37d0485bb7d (127.18.71.65:32967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b2ceadf004c475dbb36fd9ae25c7bda: Network error: Client connection negotiation failed: client connection to 127.18.71.65:32967: connect: Connection refused (error 111)
I20260809 07:48:42.123782 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24020
I20260809 07:48:42.130926 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 23640
2026-08-09T07:48:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (994 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 07:48:42.154222 18717 mini_oidc.cc:152] Starting JWKS server
I20260809 07:48:42.155848 18717 webserver.cc:460] Webserver started at https://127.0.0.1:33871/ using document root <none> and password file <none>
I20260809 07:48:42.155988 18717 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 07:48:42.156219 18717 webserver.cc:460] Webserver started at http://127.0.0.1:42873/ using document root <none> and password file <none>
2026-08-09T07:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:42Z Disabled control of system clock
I20260809 07:48:42.166167 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:39585
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:39585
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33871/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/testchainca.cert with env {}
W20260809 07:48:42.294678 24210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:42.294981 24210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:42.295081 24210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:42.299115 24210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:42.299196 24210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:42.299224 24210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:42.299274 24210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:42.299325 24210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 07:48:42.299363 24210 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 07:48:42.299393 24210 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33871/jwks.json
I20260809 07:48:42.304793 24210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:39585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:39585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33871/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:42.305881 24210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:42.306939 24210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:42.315392 24220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:42.315410 24217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:42.315553 24210 server_base.cc:1034] running on GCE node
W20260809 07:48:42.315410 24218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:42.316131 24210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:42.317246 24210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:42.318405 24210 hybrid_clock.cc:648] HybridClock initialized: now 1786261722318383 us; error 34 us; skew 500 ppm
I20260809 07:48:42.320202 24210 webserver.cc:460] Webserver started at http://127.18.71.126:37763/ using document root <none> and password file <none>
I20260809 07:48:42.320825 24210 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:42.320889 24210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:42.321144 24210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:42.323194 24210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "bebebc6b34bc45da8597e4f9bf3f268f"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "8b547df672e84c1a4d79b70dbcc40847"
server_key_iv: "8b04d107ca75e207eb391113d6ed6b96"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.323787 24210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "bebebc6b34bc45da8597e4f9bf3f268f"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "8b547df672e84c1a4d79b70dbcc40847"
server_key_iv: "8b04d107ca75e207eb391113d6ed6b96"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.327494 24210 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:48:42.329972 24226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:42.331166 24210 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:42.331301 24210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "bebebc6b34bc45da8597e4f9bf3f268f"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "8b547df672e84c1a4d79b70dbcc40847"
server_key_iv: "8b04d107ca75e207eb391113d6ed6b96"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.331408 24210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:42.348436 24210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:42.349280 24210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:42.349488 24210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:42.356326 24210 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:39585
I20260809 07:48:42.356393 24286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:39585 every 8 connection(s)
I20260809 07:48:42.357403 24210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:42.361024 24287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:42.366508 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24210
I20260809 07:48:42.366659 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:42.366979 18717 external_mini_cluster.cc:1428] Setting key a17e57dc58c2663067539d2796ee226d
I20260809 07:48:42.367442 24287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f: Bootstrap starting.
I20260809 07:48:42.370155 24287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:42.371057 24287 log.cc:824] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:42.373138 24287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f: No bootstrap required, opened a new log
I20260809 07:48:42.376333 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.368432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.18.71.126:39585)
0809 07:48:42.369126 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:42.369141 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:42.369174 (+ 33us) server_negotiation.cc:407] Connection header received
0809 07:48:42.369799 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:42.369815 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:42.370250 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.370646 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:42.371107 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.372079 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.373044 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.373566 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.373755 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:42.374625 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:42.374630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:42.374643 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:42.375017 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:42.375071 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:42.375091 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:42.375205 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:42.375357 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 07:48:42.375625 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":146,"threads_started":1}
I20260809 07:48:42.377173 24287 raft_consensus.cc:348] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39585 } }
I20260809 07:48:42.377417 24287 raft_consensus.cc:374] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:42.377502 24287 raft_consensus.cc:729] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bebebc6b34bc45da8597e4f9bf3f268f, State: Initialized, Role: FOLLOWER
I20260809 07:48:42.377921 24287 consensus_queue.cc:260] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39585 } }
I20260809 07:48:42.378069 24287 raft_consensus.cc:388] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:42.378154 24287 raft_consensus.cc:482] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:42.378253 24287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:42.379177 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:48:42.379294 24287 raft_consensus.cc:504] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39585 } }
I20260809 07:48:42.379637 24287 leader_election.cc:302] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bebebc6b34bc45da8597e4f9bf3f268f; no voters:
I20260809 07:48:42.379987 24287 leader_election.cc:288] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:42.380069 24293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:42.380280 24293 raft_consensus.cc:686] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [term 1 LEADER]: Becoming Leader. State: Replica: bebebc6b34bc45da8597e4f9bf3f268f, State: Running, Role: LEADER
I20260809 07:48:42.380625 24293 consensus_queue.cc:237] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39585 } }
I20260809 07:48:42.381338 24287 sys_catalog.cc:564] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:42.381853 24293 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bebebc6b34bc45da8597e4f9bf3f268f. Latest consensus state: current_term: 1 leader_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39585 } } }
I20260809 07:48:42.381982 24293 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:42.382266 24294 sys_catalog.cc:455] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bebebc6b34bc45da8597e4f9bf3f268f" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39585 } } }
I20260809 07:48:42.382388 24294 sys_catalog.cc:458] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:42.383836 24297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:42.386864 24297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:42.392540 24297 catalog_manager.cc:1347] Generated new cluster ID: 96a440bd7d574012a0418ed36622ffcb
I20260809 07:48:42.392606 24297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:42.406373 24297 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:42.407541 24297 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:42.415547 24297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bebebc6b34bc45da8597e4f9bf3f268f: Generated new TSK 0
I20260809 07:48:42.416325 24297 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 07:48:42.444252 24320 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 07:48:42.444399 24320 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 07:48:42.445099 24291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.427288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52328 (local address 127.18.71.126:39585)
0809 07:48:42.427412 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:42.427417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:42.427598 (+ 181us) server_negotiation.cc:407] Connection header received
0809 07:48:42.427783 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:42.427788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:42.427846 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.427946 (+ 100us) server_negotiation.cc:226] Negotiated authn=JWT
0809 07:48:42.428218 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.429083 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.429955 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.430307 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.430403 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:42.430739 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 07:48:42.444696 (+ 13957us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 07:48:42.444963 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52328: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 07:48:42.445187 24291 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52328: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 07:48:42.446506 24319 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:42.427186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39585 (local address 127.0.0.1:52328)
0809 07:48:42.427523 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:42.427535 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:42.427639 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:42.427975 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:42.427977 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:42.427983 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0809 07:48:42.428115 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:42.428119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.429193 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.429196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:42.429844 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:42.429852 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.430436 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.430439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:42.430515 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:42.430558 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 07:48:42.444932 (+ 14374us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 07:48:42.445176 (+ 244us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.71.126:39585: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":115,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.18.71.126:39585: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33871/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 07:48:42.447415 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24210
2026-08-09T07:48:42Z chronyd exiting
I20260809 07:48:42.487551 18717 test_util.cc:175] -----------------------------------------------
I20260809 07:48:42.487646 18717 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786261695308648-18717-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (335 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 07:48:42.488755 18717 mini_oidc.cc:152] Starting JWKS server
I20260809 07:48:42.489645 18717 webserver.cc:460] Webserver started at https://127.0.0.1:39217/ using document root <none> and password file <none>
I20260809 07:48:42.489727 18717 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 07:48:42.489972 18717 webserver.cc:460] Webserver started at http://127.0.0.1:45463/ using document root <none> and password file <none>
2026-08-09T07:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:42Z Disabled control of system clock
I20260809 07:48:42.500356 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40645
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:44389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:40645
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39217/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/testchainca.cert with env {}
W20260809 07:48:42.614604 24326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:42.614871 24326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:42.614909 24326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:42.618916 24326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:42.618971 24326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:42.618988 24326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:42.619004 24326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:42.619019 24326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 07:48:42.619035 24326 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 07:48:42.619048 24326 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39217/jwks.json
I20260809 07:48:42.624145 24326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:40645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39217/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:42.625185 24326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:42.626214 24326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:42.634495 24332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:42.634538 24331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:42.634512 24335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:42.634745 24326 server_base.cc:1034] running on GCE node
I20260809 07:48:42.635300 24326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:42.636322 24326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:42.637511 24326 hybrid_clock.cc:648] HybridClock initialized: now 1786261722637496 us; error 31 us; skew 500 ppm
I20260809 07:48:42.639382 24326 webserver.cc:460] Webserver started at http://127.18.71.126:44809/ using document root <none> and password file <none>
I20260809 07:48:42.639943 24326 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:42.640043 24326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:42.640262 24326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:42.642318 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "2bf5e00ab21e4e05b6417423b49c34d3"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "ec190e11bedb5185a9513aab7ac181e8"
server_key_iv: "993bf1cd5f23aeaaeb25114d6b9e0af4"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.642885 24326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "2bf5e00ab21e4e05b6417423b49c34d3"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "ec190e11bedb5185a9513aab7ac181e8"
server_key_iv: "993bf1cd5f23aeaaeb25114d6b9e0af4"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.646498 24326 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:42.648872 24341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:42.650084 24326 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:42.650220 24326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "2bf5e00ab21e4e05b6417423b49c34d3"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "ec190e11bedb5185a9513aab7ac181e8"
server_key_iv: "993bf1cd5f23aeaaeb25114d6b9e0af4"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.650329 24326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:42.663931 24326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:42.664719 24326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:42.664911 24326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:42.672049 24326 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:40645
I20260809 07:48:42.672042 24399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:40645 every 8 connection(s)
I20260809 07:48:42.673092 24326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:42.676230 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24326
I20260809 07:48:42.676362 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:42.676453 24400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:42.676745 18717 external_mini_cluster.cc:1428] Setting key c633243b94f17baf837b108150ebabc2
I20260809 07:48:42.683002 24400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3: Bootstrap starting.
I20260809 07:48:42.685536 24400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:42.685767 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.678057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60328 (local address 127.18.71.126:40645)
0809 07:48:42.678721 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:42.678737 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:42.678771 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:48:42.679425 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:42.679442 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:42.679825 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.680151 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:42.680616 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.681573 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.682501 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.682963 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.683142 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:42.684015 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:42.684019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:42.684032 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:42.684402 (+ 370us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:42.684511 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:42.684542 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:42.684644 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:42.684760 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 07:48:42.685031 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":148,"threads_started":1}
I20260809 07:48:42.686339 24400 log.cc:824] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:42.688397 24400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3: No bootstrap required, opened a new log
I20260809 07:48:42.688611 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:48:42.692342 24400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40645 } }
I20260809 07:48:42.692615 24400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:42.692691 24400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bf5e00ab21e4e05b6417423b49c34d3, State: Initialized, Role: FOLLOWER
I20260809 07:48:42.693150 24400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40645 } }
I20260809 07:48:42.693292 24400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:42.693360 24400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:42.693454 24400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:42.694342 24400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40645 } }
I20260809 07:48:42.694674 24400 leader_election.cc:302] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bf5e00ab21e4e05b6417423b49c34d3; no voters:
I20260809 07:48:42.695008 24400 leader_election.cc:288] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:42.695125 24407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:42.695416 24407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [term 1 LEADER]: Becoming Leader. State: Replica: 2bf5e00ab21e4e05b6417423b49c34d3, State: Running, Role: LEADER
I20260809 07:48:42.695775 24407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40645 } }
I20260809 07:48:42.696259 24400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:42.697386 24409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bf5e00ab21e4e05b6417423b49c34d3. Latest consensus state: current_term: 1 leader_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40645 } } }
I20260809 07:48:42.697492 24409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:42.697903 24419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:42.697858 24408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bf5e00ab21e4e05b6417423b49c34d3" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40645 } } }
I20260809 07:48:42.697979 24408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:42.701000 24419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:42.706238 24419 catalog_manager.cc:1347] Generated new cluster ID: 311c501ade1e49a2b2a84b6970a73bf1
I20260809 07:48:42.706302 24419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:42.738731 24419 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:42.739672 24419 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:42.746372 24419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bf5e00ab21e4e05b6417423b49c34d3: Generated new TSK 0
I20260809 07:48:42.747076 24419 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 07:48:42.776329 24436 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 07:48:42.776425 24436 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 07:48:42.777086 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:42.761251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.18.71.126:40645)
0809 07:48:42.761412 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:42.761417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:42.761547 (+ 130us) server_negotiation.cc:407] Connection header received
0809 07:48:42.761682 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:42.761684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:42.761748 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:42.761842 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0809 07:48:42.762129 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.762656 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.763904 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.764124 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.764200 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:42.764584 (+ 384us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 07:48:42.776749 (+ 12165us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39217/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 07:48:42.776979 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60336: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39217/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 07:48:42.777166 24404 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60336: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39217/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 07:48:42.777341 24435 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:42.761082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40645 (local address 127.0.0.1:60336)
0809 07:48:42.761427 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:42.761442 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:42.761561 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:42.761863 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:42.761867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:42.761874 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 07:48:42.761999 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:42.762006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.762765 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.762769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:42.763772 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:42.763783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:42.764235 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:42.764239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:42.764334 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:42.764362 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 07:48:42.776945 (+ 12583us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39217/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 07:48:42.777136 (+ 191us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.71.126:40645: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39217/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":116,"threads_started":1}
I20260809 07:48:42.777904 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24326
2026-08-09T07:48:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (331 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 07:48:42.819963 18717 mini_oidc.cc:152] Starting JWKS server
I20260809 07:48:42.820928 18717 webserver.cc:460] Webserver started at https://127.0.0.1:33283/ using document root <none> and password file <none>
I20260809 07:48:42.821065 18717 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 07:48:42.821331 18717 webserver.cc:460] Webserver started at http://127.0.0.1:41205/ using document root <none> and password file <none>
2026-08-09T07:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:42Z Disabled control of system clock
I20260809 07:48:42.832671 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:34247
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:34247
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33283/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 07:48:42.949347 24443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:42.949613 24443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:42.949663 24443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:42.953779 24443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:42.953843 24443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:42.953859 24443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:42.953876 24443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:42.953891 24443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 07:48:42.953907 24443 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 07:48:42.953922 24443 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33283/jwks.json
I20260809 07:48:42.958739 24443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:34247
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:34247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33283/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:42.959760 24443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:42.961185 24443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:42.969354 24451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:42.969415 24448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:42.969349 24449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:42.969642 24443 server_base.cc:1034] running on GCE node
I20260809 07:48:42.970326 24443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:42.971441 24443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:42.972658 24443 hybrid_clock.cc:648] HybridClock initialized: now 1786261722972635 us; error 42 us; skew 500 ppm
I20260809 07:48:42.974566 24443 webserver.cc:460] Webserver started at http://127.18.71.126:39669/ using document root <none> and password file <none>
I20260809 07:48:42.975147 24443 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:42.975241 24443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:42.975472 24443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:42.977556 24443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "f16519c767794e228b270ab7757f3de9"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "785dddf91bc2f8a673390189b48bbd7a"
server_key_iv: "245c9f35f539c0f2a219593fef477bda"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.978127 24443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "f16519c767794e228b270ab7757f3de9"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "785dddf91bc2f8a673390189b48bbd7a"
server_key_iv: "245c9f35f539c0f2a219593fef477bda"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.981767 24443 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:42.984424 24458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:42.985675 24443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:42.985814 24443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "f16519c767794e228b270ab7757f3de9"
format_stamp: "Formatted at 2026-08-09 07:48:42 on dist-test-slave-10pc"
server_key: "785dddf91bc2f8a673390189b48bbd7a"
server_key_iv: "245c9f35f539c0f2a219593fef477bda"
server_key_version: "encryptionkey@0"
I20260809 07:48:42.985929 24443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:43.000388 24443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:43.001176 24443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:43.001375 24443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:43.008260 24443 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:34247
I20260809 07:48:43.008267 24514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:34247 every 8 connection(s)
I20260809 07:48:43.009284 24443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:43.012565 24515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:43.017907 24515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9: Bootstrap starting.
I20260809 07:48:43.019120 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24443
I20260809 07:48:43.019263 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:43.019555 18717 external_mini_cluster.cc:1428] Setting key 5277f7d331e8d28c59132ba39ea19750
I20260809 07:48:43.020572 24515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:43.021339 24515 log.cc:824] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:43.023289 24515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9: No bootstrap required, opened a new log
I20260809 07:48:43.026695 24515 raft_consensus.cc:348] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16519c767794e228b270ab7757f3de9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34247 } }
I20260809 07:48:43.026924 24515 raft_consensus.cc:374] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:43.026997 24515 raft_consensus.cc:729] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16519c767794e228b270ab7757f3de9, State: Initialized, Role: FOLLOWER
I20260809 07:48:43.027457 24515 consensus_queue.cc:260] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16519c767794e228b270ab7757f3de9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34247 } }
I20260809 07:48:43.027606 24515 raft_consensus.cc:388] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:43.027680 24515 raft_consensus.cc:482] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:43.027799 24515 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:43.028601 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:43.020665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52016 (local address 127.18.71.126:34247)
0809 07:48:43.021378 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:43.021392 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:43.021426 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:48:43.022143 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:43.022161 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:43.022510 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:43.022833 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:43.023270 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.024261 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.025228 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.025689 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.025859 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:43.026697 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:43.026702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:43.026715 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:43.027181 (+ 466us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:43.027235 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:43.027256 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:43.027357 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:43.027477 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 07:48:43.027755 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":172,"threads_started":1}
I20260809 07:48:43.028784 24515 raft_consensus.cc:504] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16519c767794e228b270ab7757f3de9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34247 } }
I20260809 07:48:43.029126 24515 leader_election.cc:302] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f16519c767794e228b270ab7757f3de9; no voters:
I20260809 07:48:43.029438 24515 leader_election.cc:288] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:43.030323 24521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:43.030517 24521 raft_consensus.cc:686] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [term 1 LEADER]: Becoming Leader. State: Replica: f16519c767794e228b270ab7757f3de9, State: Running, Role: LEADER
I20260809 07:48:43.030689 24515 sys_catalog.cc:564] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:43.030797 24521 consensus_queue.cc:237] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16519c767794e228b270ab7757f3de9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34247 } }
I20260809 07:48:43.031438 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:48:43.032516 24524 sys_catalog.cc:455] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f16519c767794e228b270ab7757f3de9. Latest consensus state: current_term: 1 leader_uuid: "f16519c767794e228b270ab7757f3de9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16519c767794e228b270ab7757f3de9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34247 } } }
I20260809 07:48:43.032670 24524 sys_catalog.cc:458] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:43.033015 24523 sys_catalog.cc:455] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f16519c767794e228b270ab7757f3de9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16519c767794e228b270ab7757f3de9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34247 } } }
I20260809 07:48:43.033116 24523 sys_catalog.cc:458] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:43.033321 24539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:43.036312 24539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:43.042133 24539 catalog_manager.cc:1347] Generated new cluster ID: c2213e02141c47e2a94503e33f9d31e5
I20260809 07:48:43.042203 24539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:43.056803 24539 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:43.057722 24539 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:43.062630 24539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f16519c767794e228b270ab7757f3de9: Generated new TSK 0
I20260809 07:48:43.063347 24539 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:48:43.075726 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:43.070463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.18.71.126:34247)
0809 07:48:43.070611 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:43.070616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:43.070795 (+ 179us) server_negotiation.cc:407] Connection header received
0809 07:48:43.070957 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:43.070959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:43.071012 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:43.071101 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0809 07:48:43.071380 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.072107 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.073434 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.073667 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.073752 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:43.074114 (+ 362us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 07:48:43.075334 (+ 1220us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 07:48:43.075580 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52026: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 07:48:43.075814 24519 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52026: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 07:48:43.075820 24549 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:43.070377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34247 (local address 127.0.0.1:52026)
0809 07:48:43.070705 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:43.070720 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:43.070837 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:43.071140 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:43.071144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:43.071152 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 07:48:43.071275 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:43.071281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.072220 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.072223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:43.073280 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:43.073290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.073799 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.073803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:43.073895 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:43.073924 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 07:48:43.075495 (+ 1571us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 07:48:43.075643 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.71.126:34247: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":107,"threads_started":1}
I20260809 07:48:43.076542 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24443
E20260809 07:48:43.087237 24550 webserver.cc:533] Webserver: sslize failed:
E20260809 07:48:43.087318 24550 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T07:48:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (298 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T07:48:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:43Z Disabled control of system clock
I20260809 07:48:43.128913 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40855
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:40855
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/testchainca.cert with env {}
W20260809 07:48:43.239046 24554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:43.239332 24554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:43.239399 24554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:43.243358 24554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:43.243438 24554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:43.243467 24554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:43.243503 24554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:43.243541 24554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 07:48:43.243572 24554 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 07:48:43.243604 24554 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 07:48:43.248860 24554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:40855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:43.249917 24554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:43.250931 24554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:43.259379 24560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:43.259352 24562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:43.259378 24559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:43.260106 24554 server_base.cc:1034] running on GCE node
I20260809 07:48:43.260723 24554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:43.261783 24554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:43.262964 24554 hybrid_clock.cc:648] HybridClock initialized: now 1786261723262948 us; error 33 us; skew 500 ppm
I20260809 07:48:43.264880 24554 webserver.cc:460] Webserver started at http://127.18.71.126:36085/ using document root <none> and password file <none>
I20260809 07:48:43.265450 24554 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:43.265535 24554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:43.265755 24554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:43.267704 24554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "64a759bfbd4847cbb50484f89549fcc9"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-10pc"
server_key: "da02b86a0d3f48202bf7a5774480bda1"
server_key_iv: "71f623949e7f1148b73170e65069f342"
server_key_version: "encryptionkey@0"
I20260809 07:48:43.268270 24554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "64a759bfbd4847cbb50484f89549fcc9"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-10pc"
server_key: "da02b86a0d3f48202bf7a5774480bda1"
server_key_iv: "71f623949e7f1148b73170e65069f342"
server_key_version: "encryptionkey@0"
I20260809 07:48:43.271799 24554 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:43.274236 24571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:43.275404 24554 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:48:43.275534 24554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "64a759bfbd4847cbb50484f89549fcc9"
format_stamp: "Formatted at 2026-08-09 07:48:43 on dist-test-slave-10pc"
server_key: "da02b86a0d3f48202bf7a5774480bda1"
server_key_iv: "71f623949e7f1148b73170e65069f342"
server_key_version: "encryptionkey@0"
I20260809 07:48:43.275642 24554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:43.297149 24554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:43.297814 24554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:43.298010 24554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:43.304790 24554 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:40855
I20260809 07:48:43.304816 24626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:40855 every 8 connection(s)
I20260809 07:48:43.305716 24554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:43.308806 24627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:43.314214 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9: Bootstrap starting.
I20260809 07:48:43.314649 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24554
I20260809 07:48:43.314781 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:43.315083 18717 external_mini_cluster.cc:1428] Setting key f02892402715620a01dd8f5d6eaa978b
I20260809 07:48:43.316982 24627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:43.317719 24627 log.cc:824] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:43.319753 24627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9: No bootstrap required, opened a new log
I20260809 07:48:43.323963 24627 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a759bfbd4847cbb50484f89549fcc9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40855 } }
I20260809 07:48:43.324177 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:43.316331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48568 (local address 127.18.71.126:40855)
0809 07:48:43.316961 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:43.316975 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:43.317008 (+ 33us) server_negotiation.cc:407] Connection header received
0809 07:48:43.317667 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:43.317684 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:43.318160 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:43.318536 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:43.318878 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.319804 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.321047 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.321515 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.321647 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:43.322467 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:43.322472 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:43.322484 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:48:43.322860 (+ 376us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:43.322914 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:48:43.322933 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:43.323008 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:43.323207 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 07:48:43.323485 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":143,"threads_started":1}
I20260809 07:48:43.324196 24627 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:43.324322 24627 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64a759bfbd4847cbb50484f89549fcc9, State: Initialized, Role: FOLLOWER
I20260809 07:48:43.324784 24627 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a759bfbd4847cbb50484f89549fcc9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40855 } }
I20260809 07:48:43.324934 24627 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:43.325040 24627 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:43.325153 24627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:43.325970 24627 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a759bfbd4847cbb50484f89549fcc9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40855 } }
I20260809 07:48:43.326335 24627 leader_election.cc:302] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64a759bfbd4847cbb50484f89549fcc9; no voters:
I20260809 07:48:43.326647 24627 leader_election.cc:288] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:43.326714 24632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:43.326930 24632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [term 1 LEADER]: Becoming Leader. State: Replica: 64a759bfbd4847cbb50484f89549fcc9, State: Running, Role: LEADER
I20260809 07:48:43.326951 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:48:43.327224 24632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a759bfbd4847cbb50484f89549fcc9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40855 } }
I20260809 07:48:43.327884 24627 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:43.328855 24633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64a759bfbd4847cbb50484f89549fcc9. Latest consensus state: current_term: 1 leader_uuid: "64a759bfbd4847cbb50484f89549fcc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a759bfbd4847cbb50484f89549fcc9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40855 } } }
I20260809 07:48:43.329026 24633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:43.329389 24634 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64a759bfbd4847cbb50484f89549fcc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64a759bfbd4847cbb50484f89549fcc9" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40855 } } }
I20260809 07:48:43.329483 24634 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:43.329430 24641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:43.332715 24641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:43.337939 24641 catalog_manager.cc:1347] Generated new cluster ID: 6b3776ef7fb9485bae3514c966265d82
I20260809 07:48:43.338009 24641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:43.350907 24641 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:43.351864 24641 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:43.356493 24641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64a759bfbd4847cbb50484f89549fcc9: Generated new TSK 0
I20260809 07:48:43.357107 24641 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:48:43.378454 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:43.365532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48582 (local address 127.18.71.126:40855)
0809 07:48:43.365666 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:43.365669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:43.365753 (+ 84us) server_negotiation.cc:407] Connection header received
0809 07:48:43.365916 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:43.365919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:43.365962 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:43.366078 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0809 07:48:43.366360 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.366880 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.368107 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.368330 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.368406 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:43.368758 (+ 352us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 07:48:43.378064 (+ 9306us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 07:48:43.378319 (+ 255us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48582: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 07:48:43.378535 24630 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48582: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 07:48:43.378620 24656 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:43.365236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:40855 (local address 127.0.0.1:48582)
0809 07:48:43.365658 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:43.365674 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:43.365794 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:43.366109 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:43.366112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:43.366121 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 07:48:43.366238 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:43.366245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.366998 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.367002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:43.367991 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:43.368001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:43.368473 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:43.368477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:43.368577 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:43.368605 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 07:48:43.378246 (+ 9641us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 07:48:43.378389 (+ 143us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.18.71.126:40855: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":239,"threads_started":1}
I20260809 07:48:43.379313 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24554
2026-08-09T07:48:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (281 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (128 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:43 dist-test-slave-10pc krb5kdc[24670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:43 dist-test-slave-10pc krb5kdc[24670](info): set up 2 sockets
Aug 09 07:48:43 dist-test-slave-10pc krb5kdc[24670](info): commencing operation
krb5kdc: starting...
W20260809 07:48:44.719782 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:44 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:44.853313 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45781
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:45781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:44.968536 24685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:44.968840 24685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:44.968921 24685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:44.972873 24685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:44.972957 24685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:44.973006 24685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:44.973057 24685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:44.973095 24685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:44.978403 24685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:45781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:44.979458 24685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:44.980547 24685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:44.988915 24690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:44.988905 24693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:44.988905 24691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:44.989315 24685 server_base.cc:1034] running on GCE node
I20260809 07:48:44.989849 24685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:44.990899 24685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:44.992089 24685 hybrid_clock.cc:648] HybridClock initialized: now 1786261724992067 us; error 32 us; skew 500 ppm
Aug 09 07:48:44 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261724, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:44.994836 24685 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:44.995704 24685 webserver.cc:460] Webserver started at http://127.18.71.126:45119/ using document root <none> and password file <none>
I20260809 07:48:44.996263 24685 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:44.996351 24685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:44.996636 24685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:44.998688 24685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "6dbdad3efd20417db9ab62b0d3c895b5"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-10pc"
server_key: "6d48a18d754115ee4f77fbfd7dae67f8"
server_key_iv: "35598de121d12c64e43fbdea01b69b21"
server_key_version: "encryptionkey@0"
I20260809 07:48:44.999260 24685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "6dbdad3efd20417db9ab62b0d3c895b5"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-10pc"
server_key: "6d48a18d754115ee4f77fbfd7dae67f8"
server_key_iv: "35598de121d12c64e43fbdea01b69b21"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.002980 24685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:48:45.005419 24700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.006680 24685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:45.006925 24685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "6dbdad3efd20417db9ab62b0d3c895b5"
format_stamp: "Formatted at 2026-08-09 07:48:44 on dist-test-slave-10pc"
server_key: "6d48a18d754115ee4f77fbfd7dae67f8"
server_key_iv: "35598de121d12c64e43fbdea01b69b21"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.007043 24685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:45.047698 24685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:45.050796 24685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:45.050999 24685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:45.057865 24685 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:45781
I20260809 07:48:45.057898 24757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:45781 every 8 connection(s)
I20260809 07:48:45.058812 24685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:45.060725 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24685
I20260809 07:48:45.060854 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:45.061173 18717 external_mini_cluster.cc:1428] Setting key 47628ba75f6b3fc4655dd1d757844dd2
I20260809 07:48:45.061897 24758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:45.067618 24758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5: Bootstrap starting.
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261724, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:45.070007 24758 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:45.070839 24758 log.cc:824] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:45.073020 24758 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5: No bootstrap required, opened a new log
I20260809 07:48:45.075270 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.062587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33684 (local address 127.18.71.126:45781)
0809 07:48:45.063059 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.063074 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.063110 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:48:45.063740 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.063765 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.064084 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.064412 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.065239 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.066207 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.067060 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.067532 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.067702 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.070185 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:45.070190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:45.070202 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:45.070228 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:45.072164 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.072659 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.072665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.072671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.072816 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.073109 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.073112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.073114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.073606 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:45.073813 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:45.074015 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 07:48:45.074305 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":145,"threads_started":1}
I20260809 07:48:45.077417 24758 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45781 } }
I20260809 07:48:45.077679 24758 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:45.077734 24758 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dbdad3efd20417db9ab62b0d3c895b5, State: Initialized, Role: FOLLOWER
I20260809 07:48:45.078137 24758 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45781 } }
I20260809 07:48:45.078254 24758 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:45.078300 24758 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:45.078368 24758 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:45.079162 24758 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45781 } }
I20260809 07:48:45.079449 24758 leader_election.cc:302] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dbdad3efd20417db9ab62b0d3c895b5; no voters:
I20260809 07:48:45.079687 24758 leader_election.cc:288] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:45.079838 24763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:45.080124 24763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [term 1 LEADER]: Becoming Leader. State: Replica: 6dbdad3efd20417db9ab62b0d3c895b5, State: Running, Role: LEADER
I20260809 07:48:45.080529 24763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45781 } }
I20260809 07:48:45.080983 24758 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:45.082137 24764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dbdad3efd20417db9ab62b0d3c895b5. Latest consensus state: current_term: 1 leader_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45781 } } }
I20260809 07:48:45.082260 24764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:45.082561 24765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbdad3efd20417db9ab62b0d3c895b5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45781 } } }
I20260809 07:48:45.082674 24765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:45.082746 24773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:45.085704 24773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:45.090632 24773 catalog_manager.cc:1347] Generated new cluster ID: 1186314dc00f43989d8b4ac582d414d0
I20260809 07:48:45.090696 24773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:45.099707 24773 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:45.100822 24773 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:45.109413 24773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dbdad3efd20417db9ab62b0d3c895b5: Generated new TSK 0
I20260809 07:48:45.110152 24773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:45.147858 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45781
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:45.283588 24785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:45.283838 24785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:45.283881 24785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:45.287863 24785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:45.287919 24785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:45.287981 24785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:45.293102 24785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:45.294082 24785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:45.295044 24785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:45.304744 24791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.304744 24790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.304769 24793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:45.305016 24785 server_base.cc:1034] running on GCE node
I20260809 07:48:45.305428 24785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:45.306059 24785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:45.307219 24785 hybrid_clock.cc:648] HybridClock initialized: now 1786261725307201 us; error 26 us; skew 500 ppm
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.310082 24785 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:45.310999 24785 webserver.cc:460] Webserver started at http://127.18.71.65:43311/ using document root <none> and password file <none>
I20260809 07:48:45.311560 24785 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:45.311622 24785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:45.311786 24785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:45.313856 24785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "48ffdd3ea29047548b8f9f99c5bb3ac4"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "731605991aaca4418253216a8da25148"
server_key_iv: "d2286c62a0c94650f1f1154c39bd48fe"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.314379 24785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "48ffdd3ea29047548b8f9f99c5bb3ac4"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "731605991aaca4418253216a8da25148"
server_key_iv: "d2286c62a0c94650f1f1154c39bd48fe"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.317942 24785 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:45.320278 24800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.321508 24785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:45.321636 24785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "48ffdd3ea29047548b8f9f99c5bb3ac4"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "731605991aaca4418253216a8da25148"
server_key_iv: "d2286c62a0c94650f1f1154c39bd48fe"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.321739 24785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:45.355675 24785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:45.358696 24785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:45.358904 24785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:45.359527 24785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:45.360388 24785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:45.360438 24785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.360581 24785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:45.360620 24785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.369707 24785 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:43025
I20260809 07:48:45.369738 24927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:43025 every 8 connection(s)
I20260809 07:48:45.370599 24785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:45.378037 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24785
I20260809 07:48:45.378188 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:45.378530 18717 external_mini_cluster.cc:1428] Setting key 593c2fb330868e6ba8790b40a7887b62
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:45.386382 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.372379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:41925 (local address 127.18.71.126:45781)
0809 07:48:45.372534 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.372538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.373357 (+ 819us) server_negotiation.cc:407] Connection header received
0809 07:48:45.374750 (+ 1393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.374753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.374814 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.374914 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.376097 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.376739 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.377930 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.378237 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.378351 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.381781 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:45.381789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:45.381796 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:45.381831 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:45.383849 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.384351 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.384356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.384358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.384489 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.384969 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.384981 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.384985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.385219 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:45.385311 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:45.386074 (+ 763us) server_negotiation.cc:299] Negotiation successful
0809 07:48:45.386210 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:48:45.387802 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.372767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45781 (local address 127.18.71.65:41925)
0809 07:48:45.373209 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:45.373251 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:45.374520 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:45.375092 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:45.375099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:45.375459 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:45.375934 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:45.375948 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.376862 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.376871 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:45.377775 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:45.377790 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.378416 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.378421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:45.378637 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.379349 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:45.379371 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:45.381574 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:45.383962 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:45.383968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:45.383983 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:45.384245 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:45.384608 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:45.384613 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:45.384617 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:45.384828 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:45.385401 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:45.385408 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:45.385757 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 07:48:45.386019 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 07:48:45.386357 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":118,"threads_started":1}
I20260809 07:48:45.390523 24928 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45781
I20260809 07:48:45.390803 24928 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:45.391431 24928 heartbeater.cc:511] Master 127.18.71.126:45781 requested a full tablet report, sending...
I20260809 07:48:45.393076 24719 ts_manager.cc:194] Registered new tserver with Master: 48ffdd3ea29047548b8f9f99c5bb3ac4 (127.18.71.65:43025)
I20260809 07:48:45.394202 24719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:41925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:48:45.411985 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45781
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:45.528182 24934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:45.528439 24934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:45.528512 24934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:45.532521 24934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:45.532577 24934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:45.532634 24934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:45.537846 24934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:45.538878 24934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:45.539916 24934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:45.549634 24940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.549603 24942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.549598 24939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:45.549932 24934 server_base.cc:1034] running on GCE node
I20260809 07:48:45.550420 24934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:45.551095 24934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:45.552282 24934 hybrid_clock.cc:648] HybridClock initialized: now 1786261725552265 us; error 27 us; skew 500 ppm
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.555127 24934 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:48:45.556043 24934 webserver.cc:460] Webserver started at http://127.18.71.66:43631/ using document root <none> and password file <none>
I20260809 07:48:45.556689 24934 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:45.556782 24934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:45.557009 24934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:45.559134 24934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "b67736baf6ea4aa5852b6bb95b56c10b"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "00c108f7c9ec9aa8f96049bb3c482fb3"
server_key_iv: "53af6f8178c3c50932ec6a99d69e77ca"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.559712 24934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "b67736baf6ea4aa5852b6bb95b56c10b"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "00c108f7c9ec9aa8f96049bb3c482fb3"
server_key_iv: "53af6f8178c3c50932ec6a99d69e77ca"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.563437 24934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:48:45.565989 24949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.567234 24934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:45.567369 24934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b67736baf6ea4aa5852b6bb95b56c10b"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "00c108f7c9ec9aa8f96049bb3c482fb3"
server_key_iv: "53af6f8178c3c50932ec6a99d69e77ca"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.567484 24934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:45.584985 24934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:45.588310 24934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:45.588578 24934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:45.589236 24934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:45.590116 24934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:45.590185 24934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.590255 24934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:45.590301 24934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.599687 24934 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:44887
I20260809 07:48:45.599758 25070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:44887 every 8 connection(s)
I20260809 07:48:45.600729 24934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:45.609794 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 24934
I20260809 07:48:45.609931 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:45.610289 18717 external_mini_cluster.cc:1428] Setting key 2aeb22dde3c6b082d34a639116620599
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:45.615530 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.602714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:46649 (local address 127.18.71.126:45781)
0809 07:48:45.602850 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.602854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.603720 (+ 866us) server_negotiation.cc:407] Connection header received
0809 07:48:45.605259 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.605261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.605311 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.605375 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.606526 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.607051 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.608177 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.608396 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.608531 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.611445 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:45.611447 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:45.611449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:45.611474 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:45.613313 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.613836 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.613842 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.613847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.613946 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.614372 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.614375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.614376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.614555 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:45.614634 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:45.615313 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 07:48:45.615414 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 07:48:45.617014 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.603015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45781 (local address 127.18.71.66:46649)
0809 07:48:45.603483 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:45.603568 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:45.605045 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:45.605557 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:45.605564 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:45.605919 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:45.606404 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:45.606418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.607169 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.607172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:45.608073 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:45.608082 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.608585 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.608589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:45.608765 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.609434 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:45.609452 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:45.611290 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:45.613444 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:45.613450 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:45.613463 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:45.613734 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:45.614042 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:45.614045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:45.614046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:45.614252 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:45.614662 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:45.614671 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:45.615033 (+ 362us) client_negotiation.cc:769] Sending connection context
0809 07:48:45.615279 (+ 246us) client_negotiation.cc:240] Negotiation successful
0809 07:48:45.615771 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":114,"threads_started":1}
I20260809 07:48:45.618189 25071 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45781
I20260809 07:48:45.618460 25071 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:45.619139 25071 heartbeater.cc:511] Master 127.18.71.126:45781 requested a full tablet report, sending...
I20260809 07:48:45.620298 24719 ts_manager.cc:194] Registered new tserver with Master: b67736baf6ea4aa5852b6bb95b56c10b (127.18.71.66:44887)
I20260809 07:48:45.620929 24719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:46649
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:48:45.640884 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45781
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:41721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:45.767372 25077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:45.767633 25077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:45.767676 25077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:45.771798 25077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:45.771865 25077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:45.771942 25077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:45.777293 25077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:41721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45781
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:45.778290 25077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:45.779260 25077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:45.789358 25085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.789358 25083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:45.789358 25082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:45.789568 25077 server_base.cc:1034] running on GCE node
I20260809 07:48:45.790087 25077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:45.790735 25077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:45.791910 25077 hybrid_clock.cc:648] HybridClock initialized: now 1786261725791888 us; error 30 us; skew 500 ppm
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.799407 25077 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:48:45.800400 25077 webserver.cc:460] Webserver started at http://127.18.71.67:33691/ using document root <none> and password file <none>
I20260809 07:48:45.801029 25077 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:45.801100 25077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:45.801282 25077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:45.803450 25077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "10181f94951549c6bab185feeac9dc6e"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "af0f21c57507f88dd942c637701ffac1"
server_key_iv: "f4dcf98f8ed1df4add117578a2196988"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.804035 25077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "10181f94951549c6bab185feeac9dc6e"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "af0f21c57507f88dd942c637701ffac1"
server_key_iv: "f4dcf98f8ed1df4add117578a2196988"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.807991 25077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:48:45.810684 25092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.812084 25077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:48:45.812204 25077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "10181f94951549c6bab185feeac9dc6e"
format_stamp: "Formatted at 2026-08-09 07:48:45 on dist-test-slave-10pc"
server_key: "af0f21c57507f88dd942c637701ffac1"
server_key_iv: "f4dcf98f8ed1df4add117578a2196988"
server_key_version: "encryptionkey@0"
I20260809 07:48:45.812299 25077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:45.847884 25077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:45.851629 25077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:45.851897 25077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:45.852608 25077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:45.853541 25077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:45.853621 25077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.853716 25077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:45.853770 25077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:45.863484 25077 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:46529
I20260809 07:48:45.863508 25211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:46529 every 8 connection(s)
I20260809 07:48:45.864486 25077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:45.868707 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 25077
I20260809 07:48:45.868835 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:45.869200 18717 external_mini_cluster.cc:1428] Setting key 85250bef5f2dd2a7f368ec1d5a35d0eb
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:45.878535 24761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.866387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:46681 (local address 127.18.71.126:45781)
0809 07:48:45.866528 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.866531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.867434 (+ 903us) server_negotiation.cc:407] Connection header received
0809 07:48:45.869105 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.869107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.869156 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.869234 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.870714 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.871407 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.872651 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.872873 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.872953 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.875259 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:45.875262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:45.875264 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:45.875287 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:45.876736 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.877194 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.877197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.877198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.877269 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.877624 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.877626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.877627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.877808 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:45.877882 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:45.878325 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 07:48:45.878418 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 07:48:45.879371 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.866822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45781 (local address 127.18.71.67:46681)
0809 07:48:45.867290 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:45.867333 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:45.868865 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:45.869425 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:45.869434 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:45.869929 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:45.870573 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:45.870595 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.871518 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.871520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:45.872542 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:45.872552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.872996 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.872998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:45.873142 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.873665 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:45.873681 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:45.875105 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:45.876833 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:45.876838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:45.876849 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:45.877096 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:45.877366 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:45.877373 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:45.877378 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:45.877542 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:45.877893 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:45.877898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:45.878127 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 07:48:45.878280 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 07:48:45.878519 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260809 07:48:45.880383 25212 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45781
I20260809 07:48:45.880664 25212 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:45.881101 25212 heartbeater.cc:511] Master 127.18.71.126:45781 requested a full tablet report, sending...
I20260809 07:48:45.882292 24719 ts_manager.cc:194] Registered new tserver with Master: 10181f94951549c6bab185feeac9dc6e (127.18.71.67:46529)
I20260809 07:48:45.882897 24719 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:46681
I20260809 07:48:45.883088 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/test-admin.keytab.
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.900967 18717 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 07:48:45.903831 18717 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5cc' of type FILE
W20260809 07:48:45.903962 25220 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:45.902433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56900 (local address 127.18.71.65:43025)
0809 07:48:45.902716 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.902722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.902748 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:48:45.902819 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.902824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.903016 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.903135 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.903626 (+ 491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56900: BlockingRecv error: recv got EOF from 127.0.0.1:56900 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":74,"threads_started":1}
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.904935 18717 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.905848 18717 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:45.915411 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.906664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56908 (local address 127.18.71.65:43025)
0809 07:48:45.906819 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.906823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.906901 (+ 78us) server_negotiation.cc:407] Connection header received
0809 07:48:45.906972 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.906974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.907026 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.907098 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.907579 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.908422 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.909306 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.909713 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.909807 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.912121 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:45.912130 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:45.912144 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:45.912166 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:45.913901 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.914277 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.914282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.914283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.914361 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.914575 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.914578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.914579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.914800 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:45.914997 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:45.915087 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 07:48:45.915254 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 07:48:45.919214 25220 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:45.918225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.18.71.65:43025)
0809 07:48:45.918351 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.918355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.918473 (+ 118us) server_negotiation.cc:407] Connection header received
0809 07:48:45.918552 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.918554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.918589 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.918657 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.919111 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56924: BlockingRecv error: recv got EOF from 127.0.0.1:56924 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 07:48:45.919257 18717 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5cc' of type FILE
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.920291 18717 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.921209 18717 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:45.929598 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.921975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56926 (local address 127.18.71.65:43025)
0809 07:48:45.922125 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.922130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.922201 (+ 71us) server_negotiation.cc:407] Connection header received
0809 07:48:45.922298 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.922300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.922333 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.922403 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.922828 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.923369 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.924081 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.924315 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.924394 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.926574 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:45.926577 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:45.926579 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:45.926601 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:45.928094 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.928437 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.928440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.928441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.928562 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.928795 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.928797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.928798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.928982 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:45.929144 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:45.929292 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 07:48:45.929415 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 07:48:45.932088 25220 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:45.931297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56930 (local address 127.18.71.65:43025)
0809 07:48:45.931418 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.931421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.931451 (+ 30us) server_negotiation.cc:407] Connection header received
0809 07:48:45.931558 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.931560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.931591 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.931653 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.931983 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56930: BlockingRecv error: recv got EOF from 127.0.0.1:56930 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 07:48:45.932219 18717 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786261695308648-18717-0/krb5kdc/krb5cc' of type FILE
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.933276 18717 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:45.934183 18717 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 07:48:45 dist-test-slave-10pc krb5kdc[24670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261725, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:48:45.942245 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:45.934989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56936 (local address 127.18.71.65:43025)
0809 07:48:45.935122 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:45.935126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:45.935190 (+ 64us) server_negotiation.cc:407] Connection header received
0809 07:48:45.935290 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:45.935292 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:45.935324 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:45.935378 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:45.935838 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.936308 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.937166 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:45.937364 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:45.937440 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:45.939587 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:45.939590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:45.939592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:45.939612 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:45.940836 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.941267 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.941269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.941270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.941336 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:45.941550 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:45.941552 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:45.941553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:45.941682 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:45.941784 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:45.941961 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 07:48:45.942039 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:48:45.943266 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24785
I20260809 07:48:45.952610 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24934
I20260809 07:48:45.959816 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 25077
I20260809 07:48:45.968516 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 24685
2026-08-09T07:48:45Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2333 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:46 dist-test-slave-10pc krb5kdc[25234](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:46 dist-test-slave-10pc krb5kdc[25234](info): set up 2 sockets
Aug 09 07:48:46 dist-test-slave-10pc krb5kdc[25234](info): commencing operation
krb5kdc: starting...
W20260809 07:48:47.057318 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:47 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab.
Entry for principal oryx/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab.
I20260809 07:48:47.183501 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:39719
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab
--principal=oryx/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:39719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:47.296197 25249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:47.296550 25249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:47.296657 25249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:47.300626 25249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:47.300711 25249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:47.300746 25249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:47.300786 25249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:47.300817 25249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:47.306030 25249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:39719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab
--principal=oryx/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:39719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:47.307092 25249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:47.308111 25249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:47.316741 25254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.316748 25255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:47.316752 25249 server_base.cc:1034] running on GCE node
W20260809 07:48:47.316766 25257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:47.317586 25249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:47.318619 25249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:47.319782 25249 hybrid_clock.cc:648] HybridClock initialized: now 1786261727319767 us; error 30 us; skew 500 ppm
Aug 09 07:48:47 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:47.322640 25249 init.cc:377] Logged in from keytab as oryx/127.18.71.126@KRBTEST.COM (short username oryx)
I20260809 07:48:47.323539 25249 webserver.cc:460] Webserver started at http://127.18.71.126:33383/ using document root <none> and password file <none>
I20260809 07:48:47.324064 25249 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:47.324123 25249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:47.324291 25249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:47.326411 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "e806a0baba384622928f82daa2c3da62"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "075988975260b3d056ebdf30bf53f409"
server_key_iv: "f5dabeec92efd5017960a7e3fd479c0e"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.326952 25249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "e806a0baba384622928f82daa2c3da62"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "075988975260b3d056ebdf30bf53f409"
server_key_iv: "f5dabeec92efd5017960a7e3fd479c0e"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.330734 25249 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:48:47.333178 25265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.334515 25249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:47.334653 25249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "e806a0baba384622928f82daa2c3da62"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "075988975260b3d056ebdf30bf53f409"
server_key_iv: "f5dabeec92efd5017960a7e3fd479c0e"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.334764 25249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:47.350379 25249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:47.353408 25249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:47.353623 25249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:47.360855 25249 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:39719
I20260809 07:48:47.361184 25320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:39719 every 8 connection(s)
I20260809 07:48:47.361879 25249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:47.364720 25321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:47.370043 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 25249
I20260809 07:48:47.370185 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:47.370198 25321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: Bootstrap starting.
I20260809 07:48:47.370474 18717 external_mini_cluster.cc:1428] Setting key 2d73a2bd784a99fa7cc1f51a9579de23
I20260809 07:48:47.372705 25321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:47.373415 25321 log.cc:824] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:47.375440 25321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: No bootstrap required, opened a new log
Aug 09 07:48:47 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:47.379913 25321 raft_consensus.cc:348] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } }
I20260809 07:48:47.380167 25321 raft_consensus.cc:374] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:47.380241 25321 raft_consensus.cc:729] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e806a0baba384622928f82daa2c3da62, State: Initialized, Role: FOLLOWER
I20260809 07:48:47.380861 25321 consensus_queue.cc:260] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } }
I20260809 07:48:47.381038 25321 raft_consensus.cc:388] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:47.381125 25321 raft_consensus.cc:482] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:47.381227 25321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:47.382088 25321 raft_consensus.cc:504] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } }
I20260809 07:48:47.382433 25321 leader_election.cc:302] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e806a0baba384622928f82daa2c3da62; no voters:
I20260809 07:48:47.382779 25321 leader_election.cc:288] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:47.382849 25326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:47.383061 25326 raft_consensus.cc:686] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 1 LEADER]: Becoming Leader. State: Replica: e806a0baba384622928f82daa2c3da62, State: Running, Role: LEADER
I20260809 07:48:47.383410 25326 consensus_queue.cc:237] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } }
I20260809 07:48:47.384096 25321 sys_catalog.cc:564] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:47.384646 25326 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e806a0baba384622928f82daa2c3da62. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } }
I20260809 07:48:47.384835 25326 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:47.385062 25327 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } }
I20260809 07:48:47.385162 25327 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:47.385403 25332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:47.385432 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:47.371714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.18.71.126:39719)
0809 07:48:47.372146 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:47.372163 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:47.372198 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:48:47.372914 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:47.372939 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:47.373345 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:47.373661 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:47.374492 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.375563 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.376710 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.377168 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.377285 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:47.380130 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:47.380135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:47.380147 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:47.380191 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:47.382107 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:47.382555 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:47.382561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:47.382568 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:47.382682 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:47.382975 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:47.382986 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:47.382988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:47.383489 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:47.383740 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:47.384041 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 07:48:47.384363 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":131,"threads_started":1}
I20260809 07:48:47.388787 25332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:47.394373 25332 catalog_manager.cc:1347] Generated new cluster ID: 96f4bd9805ba4a43bf10343a6b58a3bc
I20260809 07:48:47.394444 25332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:47.405079 25332 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:47.405982 25332 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:47.411283 25332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: Generated new TSK 0
I20260809 07:48:47.411913 25332 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab.
Entry for principal oryx/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab.
I20260809 07:48:47.446699 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39719
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab
--principal=oryx/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:47.575026 25347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:47.575284 25347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:47.575328 25347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:47.579420 25347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:47.579493 25347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:47.579587 25347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:47.585433 25347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab
--principal=oryx/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:47.586674 25347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:47.587761 25347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:47.597286 25355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.597306 25353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.597311 25352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:47.597577 25347 server_base.cc:1034] running on GCE node
I20260809 07:48:47.598024 25347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:47.598686 25347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:47.599848 25347 hybrid_clock.cc:648] HybridClock initialized: now 1786261727599832 us; error 26 us; skew 500 ppm
Aug 09 07:48:47 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:47.602617 25347 init.cc:377] Logged in from keytab as oryx/127.18.71.65@KRBTEST.COM (short username oryx)
I20260809 07:48:47.603487 25347 webserver.cc:460] Webserver started at http://127.18.71.65:36337/ using document root <none> and password file <none>
I20260809 07:48:47.604061 25347 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:47.604122 25347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:47.604372 25347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:47.606645 25347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "29fdb83dc40c4a308ef7a8b27bdae12e"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "12b22bba8ba137922718c798baf64011"
server_key_iv: "ccdcb5659aafe5223b488e9fa888d37b"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.607199 25347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "29fdb83dc40c4a308ef7a8b27bdae12e"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "12b22bba8ba137922718c798baf64011"
server_key_iv: "ccdcb5659aafe5223b488e9fa888d37b"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.610852 25347 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:47.613369 25363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.614486 25347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:47.614625 25347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "29fdb83dc40c4a308ef7a8b27bdae12e"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "12b22bba8ba137922718c798baf64011"
server_key_iv: "ccdcb5659aafe5223b488e9fa888d37b"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.614737 25347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:47.626830 25347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:47.629622 25347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:47.629817 25347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:47.630388 25347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:47.631217 25347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:47.631286 25347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.631352 25347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:47.631398 25347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.640251 25347 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:37833
I20260809 07:48:47.640277 25485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:37833 every 8 connection(s)
I20260809 07:48:47.641168 25347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:47.643813 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 25347
I20260809 07:48:47.643941 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:47.644260 18717 external_mini_cluster.cc:1428] Setting key 38980190a18b1db80d32edb290dc6a3b
Aug 09 07:48:47 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:47.656431 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:47.642911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:36683 (local address 127.18.71.126:39719)
0809 07:48:47.643088 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:47.643093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:47.643998 (+ 905us) server_negotiation.cc:407] Connection header received
0809 07:48:47.645702 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:47.645706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:47.645757 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:47.645843 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:47.646976 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.647835 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.649677 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.649911 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.649990 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:47.652849 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:47.652852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:47.652854 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:47.652878 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:47.654316 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:47.654831 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:47.654834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:47.654835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:47.654912 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:47.655246 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:47.655249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:47.655251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:47.655469 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:47.655571 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:47.656087 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 07:48:47.656261 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 07:48:47.657193 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:47.643320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:36683)
0809 07:48:47.643831 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:47.643893 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:47.645449 (+ 1556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:47.646005 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:47.646012 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:47.646361 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:47.646829 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:47.646842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.647976 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.647983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:47.649562 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:47.649575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.650068 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.650075 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:47.650309 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:47.650923 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:47.650949 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:47.652683 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:47.654435 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:47.654441 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:47.654456 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:47.654719 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:47.655000 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:47.655003 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:47.655005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:47.655148 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:47.655572 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:47.655580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:47.655848 (+ 268us) client_negotiation.cc:769] Sending connection context
0809 07:48:47.656030 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 07:48:47.656356 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":156,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.66@KRBTEST.COM" created.
I20260809 07:48:47.658530 25486 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39719
I20260809 07:48:47.658793 25486 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:47.659303 25486 heartbeater.cc:511] Master 127.18.71.126:39719 requested a full tablet report, sending...
I20260809 07:48:47.660929 25284 ts_manager.cc:194] Registered new tserver with Master: 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833)
I20260809 07:48:47.662032 25284 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:36683
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab.
Entry for principal oryx/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab.
I20260809 07:48:47.673996 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39719
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab
--principal=oryx/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:47.786397 25491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:47.786723 25491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:47.786801 25491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:47.790805 25491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:47.790890 25491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:47.790982 25491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:47.796054 25491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab
--principal=oryx/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:47.797137 25491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:47.798137 25491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:47.807581 25497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.807600 25500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:47.807591 25498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:47.807879 25491 server_base.cc:1034] running on GCE node
I20260809 07:48:47.808341 25491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:47.809022 25491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:47.810204 25491 hybrid_clock.cc:648] HybridClock initialized: now 1786261727810186 us; error 37 us; skew 500 ppm
Aug 09 07:48:47 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:47.813072 25491 init.cc:377] Logged in from keytab as oryx/127.18.71.66@KRBTEST.COM (short username oryx)
I20260809 07:48:47.813933 25491 webserver.cc:460] Webserver started at http://127.18.71.66:46587/ using document root <none> and password file <none>
I20260809 07:48:47.814493 25491 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:47.814582 25491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:47.814807 25491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:47.816875 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "9fec0adaae5d4abd836e0233375d59a1"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "4a8247a2e754409310d3f03cf96b44cb"
server_key_iv: "451d33dc900272691202b674d85a3973"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.817467 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "9fec0adaae5d4abd836e0233375d59a1"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "4a8247a2e754409310d3f03cf96b44cb"
server_key_iv: "451d33dc900272691202b674d85a3973"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.821205 25491 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:47.823570 25507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.824985 25491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:47.825225 25491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "9fec0adaae5d4abd836e0233375d59a1"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "4a8247a2e754409310d3f03cf96b44cb"
server_key_iv: "451d33dc900272691202b674d85a3973"
server_key_version: "encryptionkey@0"
I20260809 07:48:47.825353 25491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:47.846076 25491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:47.849340 25491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:47.849547 25491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:47.850133 25491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:47.851002 25491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:47.851094 25491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.851188 25491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:47.851233 25491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:47.860080 25491 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:35097
I20260809 07:48:47.860101 25626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:35097 every 8 connection(s)
I20260809 07:48:47.861034 25491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
Aug 09 07:48:47 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:47.870780 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 25491
I20260809 07:48:47.870908 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:47.871215 18717 external_mini_cluster.cc:1428] Setting key 60a86d88cd7e6ab93af9da16d3416ee1
I20260809 07:48:47.874958 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:47.862826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:58427 (local address 127.18.71.126:39719)
0809 07:48:47.862962 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:47.862965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:47.863809 (+ 844us) server_negotiation.cc:407] Connection header received
0809 07:48:47.865219 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:47.865223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:47.865281 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:47.865381 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:47.866532 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.867064 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.868191 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.868418 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.868564 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:47.871063 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:47.871067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:47.871069 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:47.871094 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:47.872727 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:47.873339 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:47.873342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:47.873343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:47.873429 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:47.873833 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:47.873835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:47.873836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:47.874008 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:47.874079 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:47.874712 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 07:48:47.874815 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:48:47.875959 25629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:47.863235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:58427)
0809 07:48:47.863666 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:47.863707 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:47.864994 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:47.865553 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:47.865560 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:47.865904 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:47.866369 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:47.866381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.867190 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.867194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:47.868058 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:47.868066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:47.868615 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:47.868620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:47.868784 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:47.869341 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:47.869357 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:47.870876 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:47.872842 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:47.872849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:47.872864 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:47.873241 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:47.873528 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:47.873533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:47.873538 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:47.873733 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:47.874106 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:47.874115 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:47.874446 (+ 331us) client_negotiation.cc:769] Sending connection context
0809 07:48:47.874665 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 07:48:47.875062 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":104,"threads_started":1}
I20260809 07:48:47.877174 25627 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39719
I20260809 07:48:47.877435 25627 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:47.878069 25627 heartbeater.cc:511] Master 127.18.71.126:39719 requested a full tablet report, sending...
I20260809 07:48:47.879252 25284 ts_manager.cc:194] Registered new tserver with Master: 9fec0adaae5d4abd836e0233375d59a1 (127.18.71.66:35097)
I20260809 07:48:47.879822 25284 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:58427
WARNING: no policy specified for oryx/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab.
Entry for principal oryx/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab.
I20260809 07:48:47.899886 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39719
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab
--principal=oryx/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:48.014515 25632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:48.014767 25632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:48.014808 25632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:48.018884 25632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:48.018942 25632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:48.019037 25632 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:48.024185 25632 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab
--principal=oryx/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:39719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:48.025274 25632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:48.026292 25632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:48.036123 25640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.036135 25637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.036131 25638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.036415 25632 server_base.cc:1034] running on GCE node
I20260809 07:48:48.036911 25632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.037529 25632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.038820 25632 hybrid_clock.cc:648] HybridClock initialized: now 1786261728038799 us; error 59 us; skew 500 ppm
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:48.041659 25632 init.cc:377] Logged in from keytab as oryx/127.18.71.67@KRBTEST.COM (short username oryx)
I20260809 07:48:48.042559 25632 webserver.cc:460] Webserver started at http://127.18.71.67:44221/ using document root <none> and password file <none>
I20260809 07:48:48.043118 25632 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.043231 25632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.043459 25632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:48.045604 25632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "f00cd805fbc448f8acedd458bc71d24f"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a46d0bb9ccf2fe3c74a12b1a0249aa0d"
server_key_iv: "e7881c5b76a30df6300ceaf790b4b3c5"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.046193 25632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "f00cd805fbc448f8acedd458bc71d24f"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a46d0bb9ccf2fe3c74a12b1a0249aa0d"
server_key_iv: "e7881c5b76a30df6300ceaf790b4b3c5"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.049921 25632 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:48.052304 25648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.053526 25632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:48:48.053671 25632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "f00cd805fbc448f8acedd458bc71d24f"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a46d0bb9ccf2fe3c74a12b1a0249aa0d"
server_key_iv: "e7881c5b76a30df6300ceaf790b4b3c5"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.053783 25632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.064442 25632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:48.067963 25632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:48.068167 25632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.068815 25632 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:48.069669 25632 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:48.069718 25632 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.069799 25632 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:48.069831 25632 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.078131 25632 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:33023
I20260809 07:48:48.078208 25767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:33023 every 8 connection(s)
I20260809 07:48:48.079044 25632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:48.086611 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 25632
I20260809 07:48:48.086799 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:48.087146 18717 external_mini_cluster.cc:1428] Setting key 8e472193e6d8d4165e8b013028638027
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:48.094803 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.080942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:49157 (local address 127.18.71.126:39719)
0809 07:48:48.081120 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.081126 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.082148 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 07:48:48.083714 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.083719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.083786 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.083867 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.085583 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.086130 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.087874 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.088119 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.088235 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.091157 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.091160 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.091162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.091189 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.092738 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.093278 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.093282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.093283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.093375 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.093708 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.093711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.093712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.093906 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.093988 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.094536 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.094667 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 07:48:48.095629 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.081345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:49157)
0809 07:48:48.081880 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.082004 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.083431 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.084098 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.084106 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.084644 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.085311 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.085330 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.086299 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.086303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.087743 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.087755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.088326 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.088331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.088580 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.089339 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.089365 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.090979 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.092853 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.092859 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.092873 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.093146 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.093467 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.093470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.093471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.093612 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.093996 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.094001 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.094272 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.094443 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.094724 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":132,"threads_started":1}
I20260809 07:48:48.096845 25768 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39719
I20260809 07:48:48.097149 25768 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:48.097716 25768 heartbeater.cc:511] Master 127.18.71.126:39719 requested a full tablet report, sending...
I20260809 07:48:48.098820 25284 ts_manager.cc:194] Registered new tserver with Master: f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023)
I20260809 07:48:48.099407 25284 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:49157
I20260809 07:48:48.101467 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.71.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.125.keytab.
Entry for principal oryx/127.18.71.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.125.keytab.
I20260809 07:48:48.131426 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:44245
--webserver_interface=127.18.71.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.125.keytab
--principal=oryx/127.18.71.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:39719,127.18.71.125:44245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:48.249666 25773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:48.249918 25773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:48.249994 25773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:48.254040 25773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:48.254101 25773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:48.254120 25773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:48.254137 25773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:48.254153 25773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:48.259439 25773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:39719,127.18.71.125:44245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.125.keytab
--principal=oryx/127.18.71.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:44245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.71.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
W20260809 07:48:48.259644 25773 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.18.71.126:39719,127.18.71.125:44245'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 07:48:48.260583 25773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:48.261606 25773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:48.269712 25781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.269783 25778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.269721 25779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.270002 25773 server_base.cc:1034] running on GCE node
I20260809 07:48:48.270604 25773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.271737 25773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.272915 25773 hybrid_clock.cc:648] HybridClock initialized: now 1786261728272897 us; error 31 us; skew 500 ppm
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:48.275632 25773 init.cc:377] Logged in from keytab as oryx/127.18.71.125@KRBTEST.COM (short username oryx)
I20260809 07:48:48.276546 25773 webserver.cc:460] Webserver started at http://127.18.71.125:43947/ using document root <none> and password file <none>
I20260809 07:48:48.277100 25773 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.277189 25773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.277408 25773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:48.279336 25773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/instance:
uuid: "2d2a5e802b2046228b385fbff6840a0e"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "226a49ae0ac5ed604888d6a2c24ee863"
server_key_iv: "060f3aced0084c3720e32f9e9d20e5e0"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.279925 25773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal/instance:
uuid: "2d2a5e802b2046228b385fbff6840a0e"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "226a49ae0ac5ed604888d6a2c24ee863"
server_key_iv: "060f3aced0084c3720e32f9e9d20e5e0"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.283603 25773 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:48:48.286259 25790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.287458 25773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:48.287588 25773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
uuid: "2d2a5e802b2046228b385fbff6840a0e"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "226a49ae0ac5ed604888d6a2c24ee863"
server_key_iv: "060f3aced0084c3720e32f9e9d20e5e0"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.287693 25773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.307586 25773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:48.310616 25773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:48.310813 25773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.317909 25773 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.125:44245
I20260809 07:48:48.317934 25843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.125:44245 every 8 connection(s)
I20260809 07:48:48.318967 25773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
I20260809 07:48:48.321848 25844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:48.325709 25844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:48:48.328779 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 25773
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.125@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.71.125@KRBTEST.COM
I20260809 07:48:48.340752 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.327628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53236 (local address 127.18.71.126:39719)
0809 07:48:48.327805 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.327810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.328723 (+ 913us) server_negotiation.cc:407] Connection header received
0809 07:48:48.330060 (+ 1337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.330069 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.330119 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.330165 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.331499 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.332052 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.333306 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.333657 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.333788 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.336438 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.336441 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.336443 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.336514 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.338053 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.338771 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.338774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.338775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.338889 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.339356 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.339359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.339361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.339692 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.339796 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.340409 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.340573 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 07:48:48.341779 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.328007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53236)
0809 07:48:48.328547 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.328608 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.329775 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.330462 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.330468 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.330800 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.331345 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.331359 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.332185 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.332188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.333184 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.333192 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.333819 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.333823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.334016 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.334718 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.334733 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.336286 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.338174 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.338183 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.338193 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.338659 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.338996 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.339000 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.339003 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.339193 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.339804 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.339811 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.340141 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.340356 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.340707 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":56,"thread_start_us":106,"threads_started":1}
I20260809 07:48:48.341779 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.329735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48152 (local address 127.18.71.125:44245)
0809 07:48:48.330158 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.330172 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.330263 (+ 91us) server_negotiation.cc:407] Connection header received
0809 07:48:48.330445 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.330449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.330622 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.330791 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.331597 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.332755 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.333770 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.334326 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.334404 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.337181 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.337186 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.337196 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.337217 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.339184 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.339733 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.339737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.339738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.339841 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.340104 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.340107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.340108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.340378 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.340587 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.340827 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.340956 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":327,"thread_start_us":157,"threads_started":1}
I20260809 07:48:48.343250 25844 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.125@KRBTEST.COM for oryx/127.18.71.125@KRBTEST.COM
I20260809 07:48:48.351485 25846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.343797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.0.0.1:48168)
0809 07:48:48.343899 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.343916 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.343985 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.344214 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.344217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.344359 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.344447 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.344490 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.345824 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.345826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.346334 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.346341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.346742 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.346744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.346818 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.347222 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.347232 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.348398 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.350101 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.350104 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.350106 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.350293 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.350549 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.350552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.350554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.350638 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.350984 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.350986 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.351066 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.351171 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.351310 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 07:48:48.351485 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.343856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.18.71.125:44245)
0809 07:48:48.343979 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.343983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.343998 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:48.344061 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.344064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.344109 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.344187 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.344587 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.345705 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.346428 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.346653 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.346726 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.348592 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.348595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.348597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.348616 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.349978 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.350385 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.350389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.350391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.350461 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.350729 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.350733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.350735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.350893 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.350994 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.351224 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.351363 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":40}
I20260809 07:48:48.354411 25844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Bootstrap starting.
I20260809 07:48:48.356797 25844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:48.357489 25844 log.cc:824] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:48.359364 25844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: No bootstrap required, opened a new log
I20260809 07:48:48.363116 25844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } }
I20260809 07:48:48.363353 25844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:48.363404 25844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d2a5e802b2046228b385fbff6840a0e, State: Initialized, Role: FOLLOWER
I20260809 07:48:48.363874 25844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } }
I20260809 07:48:48.364357 25850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } } }
I20260809 07:48:48.364511 25850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:48.364884 25850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:48.365033 25850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } }
I20260809 07:48:48.365245 25844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:48.365875 25850 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719)
I20260809 07:48:48.366526 25294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d2a5e802b2046228b385fbff6840a0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e806a0baba384622928f82daa2c3da62" is_pre_election: true
I20260809 07:48:48.367277 25794 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 2d2a5e802b2046228b385fbff6840a0e; no voters: e806a0baba384622928f82daa2c3da62
I20260809 07:48:48.367574 25850 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 07:48:48.370678 25862 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:48.370838 25862 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:48.372372 25773 master_runner.cc:428] Detected that this master 2d2a5e802b2046228b385fbff6840a0e is joining an existing cluster
I20260809 07:48:48.372438 25773 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.71.125:44245
I20260809 07:48:48.383738 25869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.374567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.0.0.1:48176)
0809 07:48:48.374845 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.374863 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.374984 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.375317 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.375319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.375555 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.375722 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.375734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.376713 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.376715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.377619 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.377626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.378266 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.378268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.378350 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.378887 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.378903 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.379343 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.381953 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.381956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.381958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.382325 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.382648 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.382651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.382653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.382731 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.383181 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.383184 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.383271 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.383372 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.383516 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"spinlock_wait_cycles":5120,"thread_start_us":119,"threads_started":1}
I20260809 07:48:48.383813 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.375040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53244 (local address 127.18.71.126:39719)
0809 07:48:48.375161 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.375166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.375751 (+ 585us) server_negotiation.cc:407] Connection header received
0809 07:48:48.375917 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.375920 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.375964 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.376012 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.376611 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.377423 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.378599 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.378887 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.378991 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.380321 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.380325 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.380328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.380352 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.381837 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.382439 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.382442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.382444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.382557 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.382938 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.382942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.382943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.383201 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.383305 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.383531 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.383661 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:48:48.383738 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.375274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53244)
0809 07:48:48.375658 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.375674 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.375815 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.376150 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.376152 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.376346 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.376491 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.376498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.377555 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.377558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.378458 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.378470 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.379127 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.379131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.379227 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.379706 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.379727 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.380209 (+ 482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.381946 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.381951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.381953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.382326 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.382682 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.382685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.382687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.382786 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.383285 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.383288 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.383363 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.383456 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.383561 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"spinlock_wait_cycles":2432,"thread_start_us":104,"threads_started":1}
I20260809 07:48:48.384316 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.374655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48176 (local address 127.18.71.125:44245)
0809 07:48:48.374778 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.374783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.374921 (+ 138us) server_negotiation.cc:407] Connection header received
0809 07:48:48.375077 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.375080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.375133 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.375183 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.375837 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.376608 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.377722 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.378046 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.378126 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.379479 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.379483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.379485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.379506 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.381235 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.382433 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.382435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.382437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.382553 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.382887 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.382892 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.382894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.383086 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.383192 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.383983 (+ 791us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.384103 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.126@KRBTEST.COM for oryx/127.18.71.125@KRBTEST.COM
I20260809 07:48:48.397598 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.388482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48192 (local address 127.18.71.125:44245)
0809 07:48:48.388598 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.388602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.388967 (+ 365us) server_negotiation.cc:407] Connection header received
0809 07:48:48.389132 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.389134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.389174 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.389281 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.389991 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.390509 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.391741 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.391944 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.392021 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.394381 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.394385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.394387 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.394415 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.395876 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.396346 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.396350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.396352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.396428 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.396748 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.396751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.396753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.396909 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.397012 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.397341 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.397451 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 07:48:48.397636 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.388420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.0.0.1:48192)
0809 07:48:48.388823 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.388880 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.389017 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.389314 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.389320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.389744 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.389869 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.389877 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.390618 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.390622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.391625 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.391637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.392048 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.392052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.392170 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.392709 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.392724 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.394231 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.395989 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.395999 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.396006 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.396244 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.396562 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.396566 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.396568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.396651 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.397014 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.397023 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.397146 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.397289 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.397480 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":106,"threads_started":1}
I20260809 07:48:48.399192 25284 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.71.125:44245: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e"
member_type: NON_VOTER
last_known_addr {
host: "127.18.71.125"
port: 44245
}
attrs {
promote: true
}
}
dest_uuid: "e806a0baba384622928f82daa2c3da62"
cas_config_opid_index: -1
I20260809 07:48:48.399876 25284 consensus_queue.cc:237] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } }
I20260809 07:48:48.401500 25873 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 2d2a5e802b2046228b385fbff6840a0e (127.18.71.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } }
I20260809 07:48:48.402372 25817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:48.403712 25873 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.71.125:44245
I20260809 07:48:48.403803 25817 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Refusing update from remote peer e806a0baba384622928f82daa2c3da62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 07:48:48.404424 25773 master.cc:537] Master@127.18.71.125:44245 shutting down...
I20260809 07:48:48.404539 25875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:48.404961 25874 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } }
I20260809 07:48:48.405254 25874 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
W20260809 07:48:48.407167 25267 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
WARNING: no policy specified for oryx/127.18.71.124@KRBTEST.COM; defaulting to no policy
I20260809 07:48:48.409123 25773 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:48.409392 25773 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:48.409513 25773 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: stopping tablet replica
W20260809 07:48:48.409847 25267 consensus_peers.cc:487] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 -> Peer 2d2a5e802b2046228b385fbff6840a0e (127.18.71.125:44245): Couldn't send request to peer 2d2a5e802b2046228b385fbff6840a0e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.124@KRBTEST.COM" created.
I20260809 07:48:48.414932 25773 master.cc:552] Master@127.18.71.125:44245 shutdown complete.
I20260809 07:48:48.414988 25773 master_runner.cc:305] Clearing existing system tablet
I20260809 07:48:48.415262 25773 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.415330 25773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.416939 25773 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 07:48:48.418087 25880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.418344 25773 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.418715 25773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:48.418816 25773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
uuid: "2d2a5e802b2046228b385fbff6840a0e"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "226a49ae0ac5ed604888d6a2c24ee863"
server_key_iv: "060f3aced0084c3720e32f9e9d20e5e0"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.421545 25773 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:48.425253 25773 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:48:48.425361 25773 log.cc:1196] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 07:48:48.425654 25773 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Deleting consensus metadata
I20260809 07:48:48.425840 25773 master_runner.cc:315] Copying system tablet from 127.18.71.126:39719
I20260809 07:48:48.427203 25773 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: tablet copy: Beginning tablet copy session from remote peer at address 127.18.71.126:39719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.124.keytab.
Entry for principal oryx/127.18.71.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.124.keytab.
I20260809 07:48:48.430908 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:38191
--webserver_interface=127.18.71.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.124.keytab
--principal=oryx/127.18.71.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:48:48.436765 25887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.427817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53248)
0809 07:48:48.428115 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.428132 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.428264 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.428707 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.428710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.428922 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.429061 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.429068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.429976 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.429979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.430900 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.430910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.431704 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.431708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.431816 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.432931 (+ 1115us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.432947 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.433362 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.435151 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.435156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.435158 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.435365 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.435656 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.435660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.435662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.435748 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.436124 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.436127 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.436230 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.436331 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.436550 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
I20260809 07:48:48.436772 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.427945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53248 (local address 127.18.71.126:39719)
0809 07:48:48.428143 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.428149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.428201 (+ 52us) server_negotiation.cc:407] Connection header received
0809 07:48:48.428358 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.428360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.428402 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.428599 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.429209 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.429872 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.431176 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.431577 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.431704 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.433515 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.433518 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.433520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.433544 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.435035 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.435478 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.435482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.435483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.435567 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.435838 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.435842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.435844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.436027 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.436134 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.436403 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.436639 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 07:48:48.437673 25305 tablet_copy_service.cc:140] P e806a0baba384622928f82daa2c3da62: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2d2a5e802b2046228b385fbff6840a0e ({username='oryx', principal='oryx/127.18.71.125@KRBTEST.COM'} at 127.0.0.1:53248)
I20260809 07:48:48.437829 25305 tablet_copy_service.cc:161] P e806a0baba384622928f82daa2c3da62: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2d2a5e802b2046228b385fbff6840a0e at {username='oryx', principal='oryx/127.18.71.125@KRBTEST.COM'} at 127.0.0.1:53248: session id = 2d2a5e802b2046228b385fbff6840a0e-00000000000000000000000000000000
I20260809 07:48:48.439538 25305 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:48:48.441120 25773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:48:48.442948 25773 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: tablet copy: Starting download of 0 data blocks...
I20260809 07:48:48.443166 25773 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: tablet copy: Starting download of 1 WAL segments...
I20260809 07:48:48.444762 25773 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:48:48.446398 25305 tablet_copy_service.cc:342] P e806a0baba384622928f82daa2c3da62: Request end of tablet copy session 2d2a5e802b2046228b385fbff6840a0e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.71.125@KRBTEST.COM'} at 127.0.0.1:53248
I20260809 07:48:48.446499 25305 tablet_copy_service.cc:433] P e806a0baba384622928f82daa2c3da62: ending tablet copy session 2d2a5e802b2046228b385fbff6840a0e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2d2a5e802b2046228b385fbff6840a0e
I20260809 07:48:48.453354 25773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:48.454932 25889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.454988 25890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.455770 25773 server_base.cc:1034] running on GCE node
W20260809 07:48:48.456615 25892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.456852 25773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.457209 25773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.458376 25773 hybrid_clock.cc:648] HybridClock initialized: now 1786261728458361 us; error 32 us; skew 500 ppm
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:48.459844 25773 init.cc:377] Logged in from keytab as oryx/127.18.71.125@KRBTEST.COM (short username oryx)
I20260809 07:48:48.460307 25773 webserver.cc:460] Webserver started at http://127.18.71.125:36031/ using document root <none> and password file <none>
I20260809 07:48:48.460533 25773 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.460595 25773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.461663 25773 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:48.462563 25899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.462790 25773 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:48.462878 25773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
uuid: "2d2a5e802b2046228b385fbff6840a0e"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "226a49ae0ac5ed604888d6a2c24ee863"
server_key_iv: "060f3aced0084c3720e32f9e9d20e5e0"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.462976 25773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.474388 25773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:48.476398 25773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.481508 25773 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.125:44245
I20260809 07:48:48.482040 25773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
I20260809 07:48:48.483867 25955 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:48.484587 25954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.125:44245 every 8 connection(s)
I20260809 07:48:48.484928 25955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Bootstrap starting.
I20260809 07:48:48.493743 25955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:48.494071 25955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Bootstrap complete.
I20260809 07:48:48.494558 25955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } }
I20260809 07:48:48.494705 25955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2d2a5e802b2046228b385fbff6840a0e, State: Initialized, Role: LEARNER
I20260809 07:48:48.494828 25955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } }
I20260809 07:48:48.495170 25958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } }
I20260809 07:48:48.495326 25958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: LEARNER
I20260809 07:48:48.495330 25955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:48.498750 25969 catalog_manager.cc:1259] Loaded cluster ID: 96f4bd9805ba4a43bf10343a6b58a3bc
I20260809 07:48:48.498818 25969 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: loading cluster ID for follower catalog manager: success
I20260809 07:48:48.500283 25969 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: acquiring CA information for follower catalog manager: success
I20260809 07:48:48.501394 25969 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 07:48:48.590669 25888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:48.590950 25888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:48.590991 25888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:48.594920 25888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:48.594976 25888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:48.594995 25888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:48.595012 25888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:48.595027 25888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:48.600385 25888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.124.keytab
--principal=oryx/127.18.71.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:38191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.71.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:48.601518 25888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:48.602521 25888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:48.611241 25978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.611240 25976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.611220 25975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.611650 25888 server_base.cc:1034] running on GCE node
I20260809 07:48:48.612381 25888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.613592 25888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.614774 25888 hybrid_clock.cc:648] HybridClock initialized: now 1786261728614756 us; error 28 us; skew 500 ppm
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:48.617769 25888 init.cc:377] Logged in from keytab as oryx/127.18.71.124@KRBTEST.COM (short username oryx)
I20260809 07:48:48.618752 25888 webserver.cc:460] Webserver started at http://127.18.71.124:45277/ using document root <none> and password file <none>
I20260809 07:48:48.619376 25888 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.619477 25888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.619710 25888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:48.621882 25888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/instance:
uuid: "e850e16be7ad411bbc4a4b035272faee"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a380afe6f06c257315c9d152f27b1118"
server_key_iv: "dac509c7048c8c8d90c2b9ecc4989bf1"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.622488 25888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal/instance:
uuid: "e850e16be7ad411bbc4a4b035272faee"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a380afe6f06c257315c9d152f27b1118"
server_key_iv: "dac509c7048c8c8d90c2b9ecc4989bf1"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.626389 25888 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:48:48.629298 25985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.630927 25888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:48.631050 25888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
uuid: "e850e16be7ad411bbc4a4b035272faee"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a380afe6f06c257315c9d152f27b1118"
server_key_iv: "dac509c7048c8c8d90c2b9ecc4989bf1"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.631145 25888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.647138 25888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:48.650146 25888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:48.650309 25888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.658567 25888 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.124:38191
I20260809 07:48:48.658895 26043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.124:38191 every 8 connection(s)
I20260809 07:48:48.659752 25888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
I20260809 07:48:48.660262 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 25888
I20260809 07:48:48.663285 26044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:48.664932 25486 heartbeater.cc:503] Master 127.18.71.126:39719 was elected leader, sending a full tablet report...
I20260809 07:48:48.668663 26044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.71.124@KRBTEST.COM
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.124@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:48.676707 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.661886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53996 (local address 127.18.71.124:38191)
0809 07:48:48.662304 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.662326 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.662377 (+ 51us) server_negotiation.cc:407] Connection header received
0809 07:48:48.663226 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.663247 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.663638 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.663964 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.665158 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.666836 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.667732 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.668535 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.668712 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.671731 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.671736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.671745 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.671771 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.673669 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.674171 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.674177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.674183 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.674324 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.674623 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.674627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.674629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.675041 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.675260 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.675513 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.675804 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":150,"threads_started":1}
I20260809 07:48:48.680155 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.670762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53258 (local address 127.18.71.126:39719)
0809 07:48:48.670889 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.670893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.671136 (+ 243us) server_negotiation.cc:407] Connection header received
0809 07:48:48.671356 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.671359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.671402 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.671472 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.672287 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.673140 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.674163 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.674492 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.674596 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.676963 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.676966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.676968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.676991 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.678408 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.678914 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.678918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.678920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.678999 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.679277 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.679281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.679283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.679451 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.679564 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.679870 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.679989 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:48:48.680210 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.670539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53258)
0809 07:48:48.670995 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.671051 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.671238 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.671567 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.671575 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.671954 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.672153 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.672163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.673253 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.673256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.674040 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.674050 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.674616 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.674620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.674710 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.675264 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.675282 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.676788 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.678521 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.678531 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.678537 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.678814 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.679100 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.679103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.679105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.679190 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.679558 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.679564 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.679680 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.679815 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.679992 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":150,"threads_started":1}
I20260809 07:48:48.681236 26044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.124@KRBTEST.COM for oryx/127.18.71.125@KRBTEST.COM
I20260809 07:48:48.689841 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.681717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.0.0.1:48206)
0809 07:48:48.681838 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.681852 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.681925 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.682340 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.682343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.682496 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.682608 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.682615 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.683722 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.683725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.684274 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.684282 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.684878 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.684881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.684964 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.685423 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.685434 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.686726 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.688496 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.688502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.688504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.688707 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.688950 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.688952 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.688953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.689026 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.689450 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.689452 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.689527 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.689618 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.689722 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 07:48:48.689910 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.681847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48206 (local address 127.18.71.125:44245)
0809 07:48:48.682131 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.682134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.682146 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:48:48.682185 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.682187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.682225 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.682298 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.682718 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.683558 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.684383 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.684776 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.684863 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.686855 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.686858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.686860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.686883 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.688355 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.688791 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.688793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.688794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.688874 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.689136 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.689139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.689140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.689357 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.689461 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.689684 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.689789 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":126,"threads_started":1}
I20260809 07:48:48.690647 26044 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.124@KRBTEST.COM for oryx/127.18.71.124@KRBTEST.COM
I20260809 07:48:48.699038 26047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.691132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.0.0.1:54006)
0809 07:48:48.691266 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.691281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.691358 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.691565 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.691567 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.691730 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.691789 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.691793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.693018 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.693021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.693554 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.693561 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.694045 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.694047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.694108 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.694583 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.694595 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.695945 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.697699 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.697701 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.697702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.697887 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.698234 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.698236 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.698237 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.698312 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.698662 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.698664 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.698738 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.698815 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.698915 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 07:48:48.699268 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.691204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.18.71.124:38191)
0809 07:48:48.691323 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.691329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.691341 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:48:48.691429 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.691434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.691479 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.691555 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.691911 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.692878 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.693683 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.693905 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.694166 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.696103 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.696106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.696109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.696139 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.697597 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.698035 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.698039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.698040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.698130 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.698390 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.698392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.698393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.698580 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.698665 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.698878 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.698981 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:48:48.702414 26044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Bootstrap starting.
I20260809 07:48:48.704984 26044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:48.705806 26044 log.cc:824] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:48.707854 26044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: No bootstrap required, opened a new log
I20260809 07:48:48.711220 26044 raft_consensus.cc:348] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } }
I20260809 07:48:48.711459 26044 raft_consensus.cc:374] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:48.711558 26044 raft_consensus.cc:729] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e850e16be7ad411bbc4a4b035272faee, State: Initialized, Role: FOLLOWER
I20260809 07:48:48.712018 26044 consensus_queue.cc:260] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } }
I20260809 07:48:48.712579 26051 sys_catalog.cc:455] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } } }
I20260809 07:48:48.712716 26051 sys_catalog.cc:458] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:48.713447 26044 sys_catalog.cc:564] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:48:48.718586 26062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:48:48.718700 26062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:48:48.722273 25888 master_runner.cc:428] Detected that this master e850e16be7ad411bbc4a4b035272faee is joining an existing cluster
I20260809 07:48:48.722350 25888 master_runner.cc:432] Initiating AddMaster RPC to add 127.18.71.124:38191
I20260809 07:48:48.730479 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 25347
I20260809 07:48:48.744695 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.725653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54018 (local address 127.18.71.124:38191)
0809 07:48:48.725751 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.725755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.726037 (+ 282us) server_negotiation.cc:407] Connection header received
0809 07:48:48.726186 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.726192 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.726233 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.726279 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.726866 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.727632 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.729038 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.729353 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.729458 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.741520 (+ 12062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.741523 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.741525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.741545 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.743023 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.743443 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.743448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.743450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.743525 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.743787 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.743790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.743792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.743958 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.744059 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.744297 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.744489 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":31}
I20260809 07:48:48.745128 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.725653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.0.0.1:54018)
0809 07:48:48.725967 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.725981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.726086 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.726404 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.726407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.726656 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.726763 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.726773 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.727760 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.727762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.728693 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.728702 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.729727 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.729731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.729856 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.740942 (+ 11086us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.740954 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.741365 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.743151 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.743154 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.743156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.743353 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.743610 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.743615 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.743617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.743697 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.744048 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.744051 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.744150 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.744253 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.744909 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260809 07:48:48.745218 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:37833
--local_ip_for_outbound_sockets=127.18.71.65
--tserver_master_addrs=127.18.71.126:39719
--webserver_port=36337
--webserver_interface=127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab
--principal=oryx/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:48:48.762212 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.725083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.18.71.126:39719)
0809 07:48:48.725215 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.725219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.725661 (+ 442us) server_negotiation.cc:407] Connection header received
0809 07:48:48.725868 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.725872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.725925 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.725983 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.726853 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.727760 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.729139 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.729589 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.729694 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.758160 (+ 28466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.758165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.758169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.758206 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.760076 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.760683 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.760688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.760690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.760783 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.761136 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.761140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.761142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.761398 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.761485 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.761876 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.762027 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 07:48:48.762303 26068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.724972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.0.0.1:48212)
0809 07:48:48.725410 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.725425 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.725709 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.726121 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.726126 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.726507 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.726672 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.726681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.728502 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.728505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.737417 (+ 8912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.737453 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.738041 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.738045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.738139 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.757289 (+ 19150us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.757306 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.757835 (+ 529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.760535 (+ 2700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.760539 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.760541 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.760776 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.761186 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.761189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.761191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.761275 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.761781 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.761784 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.761874 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.761958 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.762102 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":86,"threads_started":1}
I20260809 07:48:48.762831 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.725052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.18.71.125:44245)
0809 07:48:48.725197 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.725202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.725492 (+ 290us) server_negotiation.cc:407] Connection header received
0809 07:48:48.725845 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.725848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.725904 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.725959 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.726815 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.727640 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.737588 (+ 9948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.737894 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.738005 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.758056 (+ 20051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.758061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.758064 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.758095 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.760372 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.760887 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.760890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.760892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.761067 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.761371 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.761374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.761376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.761586 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.761668 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.762536 (+ 868us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.762681 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:48:48.763468 26069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.725231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53260)
0809 07:48:48.725565 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.725581 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.725737 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.726148 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.726151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.726546 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.726695 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.726705 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.727922 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.727926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.728976 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.728990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.729866 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.729871 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.730056 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.757164 (+ 27108us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.757187 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.757884 (+ 697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.760226 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.760229 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.760232 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.760561 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.760904 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.760907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.760910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.760999 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.761555 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.761558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.761675 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.762871 (+ 1196us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.763270 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":27,"thread_start_us":125,"threads_started":1}
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261727, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.126@KRBTEST.COM for oryx/127.18.71.124@KRBTEST.COM
I20260809 07:48:48.777336 26046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.767919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54032 (local address 127.18.71.124:38191)
0809 07:48:48.768035 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.768040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.768138 (+ 98us) server_negotiation.cc:407] Connection header received
0809 07:48:48.768281 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.768284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.768333 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.768413 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.768897 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.769680 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.770895 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.771229 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.771355 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.773820 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.773824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.773827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.773852 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.775494 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.776006 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.776009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.776011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.776096 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.776415 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.776418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.776420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.776669 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.776864 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.777071 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.777191 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 07:48:48.777340 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.767752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.0.0.1:54032)
0809 07:48:48.768063 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.768081 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.768170 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.768448 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.768470 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.768667 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.768776 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.768784 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.769798 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.769802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.770769 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.770779 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.771377 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.771381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.771488 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.771981 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.771994 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.773674 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.775615 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.775619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.775621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.775904 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.776210 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.776213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.776215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.776317 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.776779 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.776783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.776889 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.777016 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.777163 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260809 07:48:48.778733 25284 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.18.71.124:38191: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e850e16be7ad411bbc4a4b035272faee"
member_type: NON_VOTER
last_known_addr {
host: "127.18.71.124"
port: 38191
}
attrs {
promote: true
}
}
dest_uuid: "e806a0baba384622928f82daa2c3da62"
cas_config_opid_index: 5
I20260809 07:48:48.779345 25284 consensus_queue.cc:237] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } }
I20260809 07:48:48.780474 25875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e850e16be7ad411bbc4a4b035272faee (127.18.71.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.781916 25875 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.18.71.124:38191
I20260809 07:48:48.782080 25875 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.782179 25875 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:48.783205 25888 master.cc:537] Master@127.18.71.124:38191 shutting down...
I20260809 07:48:48.783264 26016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:48.785035 26016 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Refusing update from remote peer e806a0baba384622928f82daa2c3da62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:48:48.786203 25874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:48:48.789256 25270 consensus_peers.cc:487] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 -> Peer e850e16be7ad411bbc4a4b035272faee (127.18.71.124:38191): Couldn't send request to peer e850e16be7ad411bbc4a4b035272faee. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 07:48:48.790616 25888 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:48.790974 25888 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:48.791169 25888 tablet_replica.cc:330] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: stopping tablet replica
I20260809 07:48:48.791877 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.780794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.0.0.1:48224)
0809 07:48:48.781090 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.781104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.781188 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.781607 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.781611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.781628 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:48.781711 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.781717 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.788712 (+ 6995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.788716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.790112 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.790125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.791413 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.791417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.791515 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.791569 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.791651 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.791735 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226}
I20260809 07:48:48.792342 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.780675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48224 (local address 127.18.71.125:44245)
0809 07:48:48.780989 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.780994 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.781271 (+ 277us) server_negotiation.cc:407] Connection header received
0809 07:48:48.781328 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.781331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.781392 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.781481 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:48.781838 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.782758 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.790300 (+ 7542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.791069 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.791186 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.791346 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.792121 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.792217 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260809 07:48:48.792853 25928 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 LEARNER]: Refusing update from remote peer e806a0baba384622928f82daa2c3da62: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 07:48:48.793318 25874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 07:48:48.795295 25928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e850e16be7ad411bbc4a4b035272faee (127.18.71.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.797292 25888 master.cc:552] Master@127.18.71.124:38191 shutdown complete.
I20260809 07:48:48.797381 25888 master_runner.cc:305] Clearing existing system tablet
I20260809 07:48:48.797667 25888 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.797765 25888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.798218 25958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e806a0baba384622928f82daa2c3da62. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.798357 25958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: LEARNER
I20260809 07:48:48.798461 25958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: NON_VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: true } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.798544 25958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: LEARNER
I20260809 07:48:48.799317 25888 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 07:48:48.800101 25874 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: attempting to promote NON_VOTER 2d2a5e802b2046228b385fbff6840a0e to VOTER
I20260809 07:48:48.800251 26075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.800503 25888 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.800649 25888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 07:48:48.800690 25888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
uuid: "e850e16be7ad411bbc4a4b035272faee"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a380afe6f06c257315c9d152f27b1118"
server_key_iv: "dac509c7048c8c8d90c2b9ecc4989bf1"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.800642 25874 consensus_queue.cc:237] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } }
I20260809 07:48:48.802397 25928 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 LEARNER]: Refusing update from remote peer e806a0baba384622928f82daa2c3da62: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 07:48:48.802727 25888 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:48.802794 25875 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 07:48:48.804034 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.801588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.0.0.1:54042)
0809 07:48:48.803845 (+ 2257us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.803935 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:38191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2167}
I20260809 07:48:48.804970 25871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.804548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.0.0.1:54058)
0809 07:48:48.804824 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.804876 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:38191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":97,"threads_started":1}
W20260809 07:48:48.805069 25270 consensus_peers.cc:487] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 -> Peer e850e16be7ad411bbc4a4b035272faee (127.18.71.124:38191): Couldn't send request to peer e850e16be7ad411bbc4a4b035272faee. Status: Network error: Client connection negotiation failed: client connection to 127.18.71.124:38191: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:48:48.805176 25888 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 07:48:48.805255 25888 log.cc:1196] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 07:48:48.805495 25888 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Deleting consensus metadata
I20260809 07:48:48.805642 25888 master_runner.cc:315] Copying system tablet from 127.18.71.126:39719
I20260809 07:48:48.806074 25875 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2d2a5e802b2046228b385fbff6840a0e (127.18.71.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.806845 25888 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: tablet copy: Beginning tablet copy session from remote peer at address 127.18.71.126:39719
I20260809 07:48:48.807591 25875 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.807706 25875 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:48.807825 25875 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.807895 25875 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:48.808776 25928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 2d2a5e802b2046228b385fbff6840a0e (127.18.71.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.810184 26073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.810638 26073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:48.820078 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.810902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.18.71.126:39719)
0809 07:48:48.811038 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.811043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.811404 (+ 361us) server_negotiation.cc:407] Connection header received
0809 07:48:48.811532 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.811535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.811588 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.811674 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.812231 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.813129 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.814515 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.814858 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.814961 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.816499 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.816503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.816506 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.816531 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.818248 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.818764 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.818768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.818770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.818861 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.819182 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.819185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.819187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.819403 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.819505 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.819809 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.819930 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 07:48:48.820142 26082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.811064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53266)
0809 07:48:48.811301 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.811317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.811436 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.811711 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.811714 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.811957 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.812108 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.812116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.813257 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.813261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.814354 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.814367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.814992 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.814996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.815115 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.815812 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.815829 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.816323 (+ 494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.818366 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.818370 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.818373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.818651 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.818962 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.818966 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.818968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.819080 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.819506 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.819510 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.819620 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.819740 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.819896 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260809 07:48:48.820825 25305 tablet_copy_service.cc:140] P e806a0baba384622928f82daa2c3da62: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e850e16be7ad411bbc4a4b035272faee ({username='oryx', principal='oryx/127.18.71.124@KRBTEST.COM'} at 127.0.0.1:53266)
I20260809 07:48:48.820919 25305 tablet_copy_service.cc:161] P e806a0baba384622928f82daa2c3da62: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e850e16be7ad411bbc4a4b035272faee at {username='oryx', principal='oryx/127.18.71.124@KRBTEST.COM'} at 127.0.0.1:53266: session id = e850e16be7ad411bbc4a4b035272faee-00000000000000000000000000000000
I20260809 07:48:48.822085 25305 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 07:48:48.823539 25888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 07:48:48.826151 25888 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: tablet copy: Starting download of 0 data blocks...
I20260809 07:48:48.826463 25888 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: tablet copy: Starting download of 1 WAL segments...
I20260809 07:48:48.828158 25888 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 07:48:48.830109 25305 tablet_copy_service.cc:342] P e806a0baba384622928f82daa2c3da62: Request end of tablet copy session e850e16be7ad411bbc4a4b035272faee-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.18.71.124@KRBTEST.COM'} at 127.0.0.1:53266
I20260809 07:48:48.830201 25305 tablet_copy_service.cc:433] P e806a0baba384622928f82daa2c3da62: ending tablet copy session e850e16be7ad411bbc4a4b035272faee-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e850e16be7ad411bbc4a4b035272faee
I20260809 07:48:48.838955 25888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:48.840816 26083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.841818 26084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.841981 25888 server_base.cc:1034] running on GCE node
W20260809 07:48:48.842038 26086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.842432 25888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.842741 25888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.843895 25888 hybrid_clock.cc:648] HybridClock initialized: now 1786261728843882 us; error 33 us; skew 500 ppm
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:48.845711 25888 init.cc:377] Logged in from keytab as oryx/127.18.71.124@KRBTEST.COM (short username oryx)
I20260809 07:48:48.846233 25888 webserver.cc:460] Webserver started at http://127.18.71.124:32851/ using document root <none> and password file <none>
I20260809 07:48:48.846449 25888 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.846499 25888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.847923 25888 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:48.849243 26093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.849550 25888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:48.849634 25888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
uuid: "e850e16be7ad411bbc4a4b035272faee"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a380afe6f06c257315c9d152f27b1118"
server_key_iv: "dac509c7048c8c8d90c2b9ecc4989bf1"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.849728 25888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.882454 25627 heartbeater.cc:503] Master 127.18.71.126:39719 was elected leader, sending a full tablet report...
W20260809 07:48:48.895807 26071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:48.896157 26071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:48.896236 26071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:48.900485 26071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:48.900570 26071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:48.900674 26071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:48.903292 25888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:48.905625 25888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.906795 26071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab
--principal=oryx/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:37833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=36337
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:48.907989 26071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:48.909430 26071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:48:48.911425 25888 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.124:38191
I20260809 07:48:48.911455 26155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.124:38191 every 8 connection(s)
I20260809 07:48:48.912073 25888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
I20260809 07:48:48.913961 26156 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:48:48.915010 26156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Bootstrap starting.
W20260809 07:48:48.922142 26160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.922300 26162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:48.922150 26159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:48.922616 26071 server_base.cc:1034] running on GCE node
I20260809 07:48:48.923048 26071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:48.923800 26071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:48.925011 26071 hybrid_clock.cc:648] HybridClock initialized: now 1786261728924994 us; error 33 us; skew 500 ppm
I20260809 07:48:48.926569 26156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:48:48.926888 26156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: Bootstrap complete.
I20260809 07:48:48.927450 26156 raft_consensus.cc:348] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } }
I20260809 07:48:48.927639 26156 raft_consensus.cc:729] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e850e16be7ad411bbc4a4b035272faee, State: Initialized, Role: LEARNER
I20260809 07:48:48.927775 26156 consensus_queue.cc:260] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } }
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:48.928330 26156 sys_catalog.cc:564] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:48.928272 26168 sys_catalog.cc:455] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:48.928694 26168 sys_catalog.cc:458] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: This master's current role is: LEARNER
I20260809 07:48:48.928759 26071 init.cc:377] Logged in from keytab as oryx/127.18.71.65@KRBTEST.COM (short username oryx)
I20260809 07:48:48.930240 26071 webserver.cc:460] Webserver started at http://127.18.71.65:36337/ using document root <none> and password file <none>
I20260809 07:48:48.930982 26071 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:48.931097 26071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:48.931576 26182 catalog_manager.cc:1259] Loaded cluster ID: 96f4bd9805ba4a43bf10343a6b58a3bc
I20260809 07:48:48.931689 26182 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: loading cluster ID for follower catalog manager: success
I20260809 07:48:48.933681 26182 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: acquiring CA information for follower catalog manager: success
I20260809 07:48:48.935298 26182 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:48:48.936090 26071 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:48.938509 26184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.939862 26071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 07:48:48.940004 26071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "29fdb83dc40c4a308ef7a8b27bdae12e"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "12b22bba8ba137922718c798baf64011"
server_key_iv: "ccdcb5659aafe5223b488e9fa888d37b"
server_key_version: "encryptionkey@0"
I20260809 07:48:48.940640 26071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:48.959631 26071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:48.963264 26071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:48.963488 26071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:48.964210 26071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:48.965199 26071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:48.965272 26071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.965344 26071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:48.965384 26071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:48.974786 26071 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:37833
I20260809 07:48:48.974810 26306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:37833 every 8 connection(s)
I20260809 07:48:48.975832 26071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:48.980862 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 26071
I20260809 07:48:48.981096 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 25491
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.124@KRBTEST.COM
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.125@KRBTEST.COM
Aug 09 07:48:48 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261728, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:48.993913 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:35097
--local_ip_for_outbound_sockets=127.18.71.66
--tserver_master_addrs=127.18.71.126:39719
--webserver_port=46587
--webserver_interface=127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab
--principal=oryx/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:48:48.995790 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.977952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:50177 (local address 127.18.71.124:38191)
0809 07:48:48.978218 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.978222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.979257 (+ 1035us) server_negotiation.cc:407] Connection header received
0809 07:48:48.980620 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.980624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.980687 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.980754 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.982685 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.983861 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.986599 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.987109 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.987238 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.991030 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.991035 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.991038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.991069 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.993048 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.993726 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.993732 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.993734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.993854 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.994358 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.994362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.994364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.994636 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.994714 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.995428 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.995569 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":91,"threads_started":1}
I20260809 07:48:48.997061 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.978497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.18.71.65:50177)
0809 07:48:48.979064 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.979142 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.980330 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.981206 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.981214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.981674 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.982487 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.982513 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.984624 (+ 2111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.984631 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.986451 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.986468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.987451 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.987458 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.987831 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.988701 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.988724 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.990820 (+ 2096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.993191 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.993200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.993225 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.993597 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.993999 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.994013 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.994016 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.994224 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.994786 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.994793 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.995159 (+ 366us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.995361 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.995721 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":62,"spinlock_wait_cycles":30720,"thread_start_us":168,"threads_started":1}
I20260809 07:48:48.998888 26308 heartbeater.cc:347] Connected to a master server at 127.18.71.124:38191
I20260809 07:48:48.999039 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.977851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:33279 (local address 127.18.71.125:44245)
0809 07:48:48.978060 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.978067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.979256 (+ 1189us) server_negotiation.cc:407] Connection header received
0809 07:48:48.980620 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.980624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.980688 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.980751 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.983406 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.984419 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.988142 (+ 3723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.988572 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.988681 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.993174 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.993182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.993189 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.993228 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.996795 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.997567 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.997573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.997574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.997674 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.998008 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.998011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.998012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.998292 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.998368 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.998722 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.998856 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 07:48:48.999150 26308 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:48.999385 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.979743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:56447 (local address 127.18.71.126:39719)
0809 07:48:48.979878 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:48.979883 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:48.979965 (+ 82us) server_negotiation.cc:407] Connection header received
0809 07:48:48.980624 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:48.980628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:48.980694 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:48.980755 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:48.988295 (+ 7540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.989283 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.990717 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.991071 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.991171 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.994676 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:48.994683 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:48.994688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:48.994715 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:48.996775 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.997645 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.997649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.997651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.997760 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:48.998146 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:48.998153 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:48.998159 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:48.998438 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:48.998529 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:48.998946 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 07:48:48.999123 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:48:48.999786 26308 heartbeater.cc:511] Master 127.18.71.124:38191 requested a full tablet report, sending...
I20260809 07:48:49.000089 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.978497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.18.71.65:33279)
0809 07:48:48.979064 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.979134 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.980330 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.982879 (+ 2549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.982882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.983111 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.983237 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.983244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.986659 (+ 3415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.986665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.987956 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.987972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.988876 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.988883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.989042 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.989839 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.989862 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.992963 (+ 3101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.996938 (+ 3975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.996941 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.996943 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.997439 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.997794 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.997799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.997800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.997893 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.998437 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.998440 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.998539 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.999277 (+ 738us) client_negotiation.cc:240] Negotiation successful
0809 07:48:48.999425 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":2,"spinlock_wait_cycles":20736,"thread_start_us":130,"threads_started":1}
I20260809 07:48:49.000538 26315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:48.979603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:56447)
0809 07:48:48.979879 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:48.979893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:48.980334 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:48.987645 (+ 7311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:48.987648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:48.987910 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:48.988099 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.988108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.989441 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.989444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.990565 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:48.990576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:48.991379 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:48.991383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:48.991557 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:48.992098 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:48.992110 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:48.994514 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:48.996930 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.996937 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.996939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.997500 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.997892 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:48.997898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:48.997904 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:48.998024 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:48.998611 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:48.998617 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:48.998735 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 07:48:48.999895 (+ 1160us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.000133 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":32,"thread_start_us":108,"threads_started":1}
I20260809 07:48:49.001178 26310 heartbeater.cc:347] Connected to a master server at 127.18.71.125:44245
I20260809 07:48:49.001296 26310 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.001502 26310 heartbeater.cc:511] Master 127.18.71.125:44245 requested a full tablet report, sending...
I20260809 07:48:49.001888 26307 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39719
I20260809 07:48:49.001968 26307 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.002131 26307 heartbeater.cc:511] Master 127.18.71.126:39719 requested a full tablet report, sending...
I20260809 07:48:49.002429 26113 ts_manager.cc:194] Registered new tserver with Master: 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833)
I20260809 07:48:49.002648 25916 ts_manager.cc:194] Registered new tserver with Master: 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833)
I20260809 07:48:49.003031 25284 ts_manager.cc:194] Re-registered known tserver with Master: 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833)
I20260809 07:48:49.004014 25284 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:56447
I20260809 07:48:49.102044 25768 heartbeater.cc:503] Master 127.18.71.126:39719 was elected leader, sending a full tablet report...
W20260809 07:48:49.117726 26316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:49.117966 26316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:49.118008 26316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:49.122068 26316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:49.122120 26316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:49.122202 26316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:49.127442 26316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab
--principal=oryx/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:35097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=46587
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:49.128618 26316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:49.129984 26316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:49.140225 26325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:49.140347 26322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:49.140281 26321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:49.140693 26316 server_base.cc:1034] running on GCE node
I20260809 07:48:49.141089 26316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:49.141769 26316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:49.142947 26316 hybrid_clock.cc:648] HybridClock initialized: now 1786261729142913 us; error 48 us; skew 500 ppm
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:49.146006 26316 init.cc:377] Logged in from keytab as oryx/127.18.71.66@KRBTEST.COM (short username oryx)
I20260809 07:48:49.146965 26316 webserver.cc:460] Webserver started at http://127.18.71.66:46587/ using document root <none> and password file <none>
I20260809 07:48:49.147554 26316 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:49.147660 26316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:49.151736 26316 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:49.153908 26332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:49.155195 26316 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:49.155328 26316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "9fec0adaae5d4abd836e0233375d59a1"
format_stamp: "Formatted at 2026-08-09 07:48:47 on dist-test-slave-10pc"
server_key: "4a8247a2e754409310d3f03cf96b44cb"
server_key_iv: "451d33dc900272691202b674d85a3973"
server_key_version: "encryptionkey@0"
I20260809 07:48:49.155943 26316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:49.167795 26316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:49.170959 26316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:49.171155 26316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:49.171764 26316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:49.172703 26316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:49.172770 26316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:49.172833 26316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:49.172878 26316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:49.182542 26316 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:35097
I20260809 07:48:49.182570 26452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:35097 every 8 connection(s)
I20260809 07:48:49.183553 26316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:49.193063 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 26316
I20260809 07:48:49.193284 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 25632
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.125@KRBTEST.COM
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.124@KRBTEST.COM
I20260809 07:48:49.201587 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.185680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:37261 (local address 127.18.71.125:44245)
0809 07:48:49.185836 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.185840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.187025 (+ 1185us) server_negotiation.cc:407] Connection header received
0809 07:48:49.188206 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.188209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.188261 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.188341 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.189597 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.190132 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.191718 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.192044 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.192156 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.196444 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.196449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.196473 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.196503 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.198083 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.199010 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.199017 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.199022 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.199139 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.199701 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.199708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.199712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.200053 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.200150 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.201147 (+ 997us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.201320 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 07:48:49.202216 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.187236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:39055 (local address 127.18.71.124:38191)
0809 07:48:49.187360 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.187365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.187407 (+ 42us) server_negotiation.cc:407] Connection header received
0809 07:48:49.188499 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.188504 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.188557 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.188644 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.189611 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.190398 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.191751 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.192002 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.192121 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.197838 (+ 5717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.197846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.197852 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.197892 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.199925 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.200437 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.200441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.200443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.200574 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.200890 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.200896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.200901 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.201134 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.201203 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.201578 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.202057 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 07:48:49.202307 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.185501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:38765 (local address 127.18.71.126:39719)
0809 07:48:49.185692 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.185698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.186821 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 07:48:49.187896 (+ 1075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.187900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.187961 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.188055 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.189559 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.190398 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.191501 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.191771 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.191879 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.198347 (+ 6468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.198352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.198356 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.198392 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.200436 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.200932 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.200938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.200942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.201023 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.201361 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.201364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.201366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.201587 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.201655 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.202025 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.202152 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 07:48:49.203562 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.185995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:38765)
0809 07:48:49.186593 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.186673 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.187693 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.188235 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.188243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.188912 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:49.189417 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.189429 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.190563 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.190566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.191379 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.191389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.192075 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.192082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.192311 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.193146 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:49.193216 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:49.197156 (+ 3940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:49.200582 (+ 3426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.200585 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.200587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.200822 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.201156 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.201162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.201166 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.201258 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.201740 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:49.201744 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:49.201828 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.202517 (+ 689us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.202662 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":43,"spinlock_wait_cycles":4992,"thread_start_us":694,"threads_started":1}
I20260809 07:48:49.203577 26460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.187059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.18.71.66:39055)
0809 07:48:49.187331 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.187343 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.187708 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.188691 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.188696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.188952 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:49.189435 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.189441 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.190517 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.190520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.191599 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.191610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.192124 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.192128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.192306 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.193228 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:49.193240 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:49.197644 (+ 4404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:49.200060 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.200063 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.200065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.200317 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.200691 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.200693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.200695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.200778 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.201272 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:49.201275 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:49.201379 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.202853 (+ 1474us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.202982 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":238,"spinlock_wait_cycles":8960,"thread_start_us":96,"threads_started":1}
I20260809 07:48:49.203913 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.185996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.18.71.66:37261)
0809 07:48:49.186914 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.186925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.187708 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.188383 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.188385 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.188945 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:49.189431 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.189441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.190294 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.190297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.191599 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.191610 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.192298 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.192301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.192381 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.193156 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:49.193192 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:49.196274 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:49.198451 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.198467 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.198486 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.198893 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.199293 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.199300 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.199305 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.199539 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.200258 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:49.200269 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:49.200755 (+ 486us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.201820 (+ 1065us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.202300 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":148,"spinlock_wait_cycles":40064,"thread_start_us":151,"threads_started":1}
I20260809 07:48:49.205432 26456 heartbeater.cc:347] Connected to a master server at 127.18.71.125:44245
I20260809 07:48:49.205691 26456 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.206211 26456 heartbeater.cc:511] Master 127.18.71.125:44245 requested a full tablet report, sending...
I20260809 07:48:49.207525 25916 ts_manager.cc:194] Registered new tserver with Master: 9fec0adaae5d4abd836e0233375d59a1 (127.18.71.66:35097)
I20260809 07:48:49.210042 26455 heartbeater.cc:347] Connected to a master server at 127.18.71.124:38191
I20260809 07:48:49.210136 26455 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.210166 26453 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39719
I20260809 07:48:49.210258 26453 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.210299 26455 heartbeater.cc:511] Master 127.18.71.124:38191 requested a full tablet report, sending...
I20260809 07:48:49.210459 26453 heartbeater.cc:511] Master 127.18.71.126:39719 requested a full tablet report, sending...
I20260809 07:48:49.211124 25284 ts_manager.cc:194] Re-registered known tserver with Master: 9fec0adaae5d4abd836e0233375d59a1 (127.18.71.66:35097)
I20260809 07:48:49.211104 26113 ts_manager.cc:194] Registered new tserver with Master: 9fec0adaae5d4abd836e0233375d59a1 (127.18.71.66:35097)
I20260809 07:48:49.211815 25284 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:38765
I20260809 07:48:49.215876 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:33023
--local_ip_for_outbound_sockets=127.18.71.67
--tserver_master_addrs=127.18.71.126:39719
--webserver_port=44221
--webserver_interface=127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab
--principal=oryx/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:48:49.319362 26462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.315777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.0.0.1:54070)
0809 07:48:49.316028 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.316046 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.316200 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.316558 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.316562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.316583 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:49.316695 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.316701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.317457 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.317462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.318246 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.318256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.318942 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.318946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.319036 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.319060 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.319131 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.319212 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":79,"threads_started":1}
I20260809 07:48:49.319360 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.315872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54070 (local address 127.18.71.124:38191)
0809 07:48:49.316074 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.316083 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.316143 (+ 60us) server_negotiation.cc:407] Connection header received
0809 07:48:49.316313 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.316316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.316377 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.316530 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:49.316810 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.317342 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.318374 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.318810 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.318930 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.319110 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.319146 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.319213 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 07:48:49.320129 26128 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 07:48:49.320550 26168 sys_catalog.cc:455] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: SysCatalogTable state changed. Reason: New leader e806a0baba384622928f82daa2c3da62. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:49.320729 26168 sys_catalog.cc:458] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: This master's current role is: LEARNER
W20260809 07:48:49.333637 26461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:49.333905 26461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:49.333985 26461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:49.338151 26461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:49.338231 26461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:49.338340 26461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:49.343819 26461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab
--principal=oryx/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:33023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=44221
--tserver_master_addrs=127.18.71.126:39719,127.18.71.125:44245,127.18.71.124:38191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:49.344961 26461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:49.346002 26461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:49.355924 26467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:49.356071 26461 server_base.cc:1034] running on GCE node
W20260809 07:48:49.355911 26468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:49.355924 26473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:49.356812 26461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:49.357683 26461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:49.358894 26461 hybrid_clock.cc:648] HybridClock initialized: now 1786261729358867 us; error 39 us; skew 500 ppm
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:49.362663 26461 init.cc:377] Logged in from keytab as oryx/127.18.71.67@KRBTEST.COM (short username oryx)
I20260809 07:48:49.363662 26461 webserver.cc:460] Webserver started at http://127.18.71.67:44221/ using document root <none> and password file <none>
I20260809 07:48:49.364245 26461 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:49.364351 26461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:49.368542 26461 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:48:49.370756 26480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:49.372143 26461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:49.372262 26461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "f00cd805fbc448f8acedd458bc71d24f"
format_stamp: "Formatted at 2026-08-09 07:48:48 on dist-test-slave-10pc"
server_key: "a46d0bb9ccf2fe3c74a12b1a0249aa0d"
server_key_iv: "e7881c5b76a30df6300ceaf790b4b3c5"
server_key_version: "encryptionkey@0"
I20260809 07:48:49.372880 26461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:49.400110 26461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:49.403643 26461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:49.403856 26461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:49.404687 26461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:49.405622 26461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:49.405694 26461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:49.405761 26461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:49.405805 26461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:49.416534 26461 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:33023
I20260809 07:48:49.416563 26600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:33023 every 8 connection(s)
I20260809 07:48:49.417562 26461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:49.422938 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 26461
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.125@KRBTEST.COM
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.124@KRBTEST.COM
Aug 09 07:48:49 dist-test-slave-10pc krb5kdc[25234](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261729, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:49.436264 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.420902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:38077 (local address 127.18.71.125:44245)
0809 07:48:49.421066 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.421072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.421946 (+ 874us) server_negotiation.cc:407] Connection header received
0809 07:48:49.423200 (+ 1254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.423204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.423265 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.423328 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.425304 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.426107 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.427862 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.428290 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.428370 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.431800 (+ 3430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.431804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.431808 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.431838 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.433647 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.434333 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.434339 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.434343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.434429 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.434862 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.434865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.434867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.435119 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.435185 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.435911 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.436039 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:48:49.437918 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.420762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.18.71.67:38077)
0809 07:48:49.421844 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.421861 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.422874 (+ 1013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.423766 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.423776 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.424328 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:49.425124 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.425148 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.426299 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.426305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.427722 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.427733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.428678 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.428683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.428879 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.429585 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:49.429614 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:49.431612 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:49.433776 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.433781 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.433797 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.434202 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.434552 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.434555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.434557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.434743 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.435270 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:49.435276 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:49.435612 (+ 336us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.436423 (+ 811us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.436787 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":9,"spinlock_wait_cycles":8960,"thread_start_us":103,"threads_started":1}
I20260809 07:48:49.439677 26603 heartbeater.cc:347] Connected to a master server at 127.18.71.125:44245
I20260809 07:48:49.439931 26603 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.440575 26603 heartbeater.cc:511] Master 127.18.71.125:44245 requested a full tablet report, sending...
I20260809 07:48:49.441882 25916 ts_manager.cc:194] Registered new tserver with Master: f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023)
I20260809 07:48:49.443495 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.421625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:51249 (local address 127.18.71.126:39719)
0809 07:48:49.421760 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.421764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.422169 (+ 405us) server_negotiation.cc:407] Connection header received
0809 07:48:49.423225 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.423235 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.423313 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.423385 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.428219 (+ 4834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.429169 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.431800 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.432487 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.432608 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.437457 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.437462 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.437467 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.437499 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.440181 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.441189 (+ 1008us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.441194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.441198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.441295 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.442413 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.442419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.442424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.442679 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.442773 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.443189 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.443310 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":51}
I20260809 07:48:49.443499 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.421526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:51249)
0809 07:48:49.422087 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.422097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.423087 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.423770 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.423777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.424918 (+ 1141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:49.425123 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.425147 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.429379 (+ 4232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.429384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.430437 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.430452 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.433881 (+ 3429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.433895 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.434082 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.434968 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:49.434988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:49.436707 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:49.440284 (+ 3577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.440289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.440292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.440580 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.441425 (+ 845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.441429 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.441433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.441523 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.442843 (+ 1320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:49.442847 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:49.442952 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.443075 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.443234 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":76,"threads_started":1}
I20260809 07:48:49.444116 26601 heartbeater.cc:347] Connected to a master server at 127.18.71.126:39719
I20260809 07:48:49.444206 26601 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.444211 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.419518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:37475 (local address 127.18.71.124:38191)
0809 07:48:49.419670 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.419675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.421569 (+ 1894us) server_negotiation.cc:407] Connection header received
0809 07:48:49.424048 (+ 2479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.424054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.424133 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.424191 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.425537 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.426487 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.430607 (+ 4120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.431076 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.431170 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.437012 (+ 5842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.437018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.437023 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.437057 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.439693 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.440719 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.440724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.440729 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.440829 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.441644 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.441649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.441654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.441931 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.442040 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.443811 (+ 1771us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.443966 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:48:49.444403 26601 heartbeater.cc:511] Master 127.18.71.126:39719 requested a full tablet report, sending...
I20260809 07:48:49.444911 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.419946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.18.71.67:37475)
0809 07:48:49.421386 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.421450 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.422861 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.424371 (+ 1510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.424376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.424653 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:49.425392 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.425402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.426896 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.426902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.428055 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.428070 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.431349 (+ 3279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.431354 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.431499 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.432171 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:49.432191 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:49.436848 (+ 4657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:49.439868 (+ 3020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.439873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.439878 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.440182 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.440948 (+ 766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:49.440954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:49.440959 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:49.441069 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:49.442113 (+ 1044us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:49.442118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:49.442227 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.444472 (+ 2245us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.444658 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1193,"mutex_wait_us":540,"thread_start_us":644,"threads_started":1}
I20260809 07:48:49.445102 25284 ts_manager.cc:194] Re-registered known tserver with Master: f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023)
I20260809 07:48:49.445778 26602 heartbeater.cc:347] Connected to a master server at 127.18.71.124:38191
I20260809 07:48:49.445876 26602 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:49.446048 25284 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:51249
I20260809 07:48:49.446072 26602 heartbeater.cc:511] Master 127.18.71.124:38191 requested a full tablet report, sending...
I20260809 07:48:49.447244 26113 ts_manager.cc:194] Registered new tserver with Master: f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023)
I20260809 07:48:49.448642 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.425251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53270 (local address 127.18.71.126:39719)
0809 07:48:49.427052 (+ 1801us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.427065 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.427388 (+ 323us) server_negotiation.cc:407] Connection header received
0809 07:48:49.427429 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.427432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.427480 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.427569 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.428603 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.430065 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.431421 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.432954 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.433069 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.438228 (+ 5159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.438232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.438235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.438261 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.441849 (+ 3588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.443899 (+ 2050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.443906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.443908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.444003 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.445440 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.445444 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.445445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.446441 (+ 996us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.446572 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.448275 (+ 1703us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.448408 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":1722,"thread_start_us":69,"threads_started":1}
I20260809 07:48:49.452349 25284 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53270:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 07:48:49.456940 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.426658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.18.71.124:38191)
0809 07:48:49.438751 (+ 12093us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.438767 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.438783 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:48:49.446416 (+ 7633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.446422 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.446491 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.448252 (+ 1761us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.448372 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.449340 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.450875 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.451358 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.451875 (+ 517us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.453085 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.453089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.453092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.453115 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.454900 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.455410 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.455413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.455415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.455497 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.455825 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.455828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.455830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.456069 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.456567 (+ 498us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.456644 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.456765 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11995,"thread_start_us":95,"threads_started":1}
W20260809 07:48:49.457708 25284 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:49.457983 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.426483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.18.71.125:44245)
0809 07:48:49.434633 (+ 8150us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.434638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.434978 (+ 340us) server_negotiation.cc:407] Connection header received
0809 07:48:49.435013 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.435015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.435056 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.435134 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.436691 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.442121 (+ 5430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.444660 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.445001 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.445100 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.451624 (+ 6524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.451629 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.451632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.451657 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.454077 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.454684 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.454690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.454694 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.454808 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.455241 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.455246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.455249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.455608 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.455844 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.457561 (+ 1717us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.457724 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7205,"thread_start_us":96,"threads_started":1}
I20260809 07:48:49.461136 25874 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: NON_VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: true } } }
I20260809 07:48:49.461273 25874 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:49.465198 25874 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: attempting to promote NON_VOTER e850e16be7ad411bbc4a4b035272faee to VOTER
I20260809 07:48:49.465561 25874 consensus_queue.cc:237] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:49.466681 25928 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Refusing update from remote peer e806a0baba384622928f82daa2c3da62: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 07:48:49.466924 26128 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 LEARNER]: Refusing update from remote peer e806a0baba384622928f82daa2c3da62: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 07:48:49.467204 25874 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 07:48:49.467378 26622 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 07:48:49.469233 26622 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e850e16be7ad411bbc4a4b035272faee (127.18.71.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.470762 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.470853 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:49.470639 26128 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e850e16be7ad411bbc4a4b035272faee (127.18.71.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.470961 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.471030 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:49.471138 26622 sys_catalog.cc:455] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.471232 26622 sys_catalog.cc:458] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:49.471879 25928 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e850e16be7ad411bbc4a4b035272faee (127.18.71.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.474079 26632 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.474278 26632 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:49.474342 26168 sys_catalog.cc:455] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e806a0baba384622928f82daa2c3da62" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:49.474460 26168 sys_catalog.cc:458] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:49.496028 26462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.478403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33023 (local address 127.0.0.1:51428)
0809 07:48:49.478517 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.478533 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.478618 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.489216 (+ 10598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.489222 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.489247 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:49.489369 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.489378 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.491203 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.491208 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.492498 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.492513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.495578 (+ 3065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.495581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.495660 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.495681 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.495752 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.495880 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":19}
I20260809 07:48:49.496078 26636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.478674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35097 (local address 127.0.0.1:60476)
0809 07:48:49.479997 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.480024 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.480117 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.488216 (+ 8099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.488220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.488238 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:49.488354 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.488361 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.489827 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.489830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.493703 (+ 3873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.493713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.495265 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.495268 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.495424 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.495832 (+ 408us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.495894 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.495955 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1261,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260809 07:48:49.496692 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.478559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.18.71.67:33023)
0809 07:48:49.488658 (+ 10099us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.488671 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.488698 (+ 27us) server_negotiation.cc:407] Connection header received
0809 07:48:49.488785 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.488792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.489003 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.489542 (+ 539us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:49.489717 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.491026 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.492644 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.493531 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.495582 (+ 2051us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.495728 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.495784 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.496439 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9964,"thread_start_us":117,"threads_started":1}
I20260809 07:48:49.497619 26634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.479212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37833 (local address 127.0.0.1:38788)
0809 07:48:49.480309 (+ 1097us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.480323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.480404 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.491500 (+ 11096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.491505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.491532 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:49.491658 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.491667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.493791 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.493796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.495545 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.495563 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.497200 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.497203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.497291 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.497322 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.497388 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.497455 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1034,"mutex_wait_us":4,"thread_start_us":93,"threads_started":1}
I20260809 07:48:49.498067 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.479387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.18.71.65:37833)
0809 07:48:49.480682 (+ 1295us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.480699 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.480724 (+ 25us) server_negotiation.cc:407] Connection header received
0809 07:48:49.480809 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.480817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.481056 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.481225 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:49.491934 (+ 10709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.493599 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.495736 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.497005 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.497213 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.497455 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.497517 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.497648 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1173,"thread_start_us":111,"threads_started":1}
I20260809 07:48:49.499567 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.479124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.18.71.66:35097)
0809 07:48:49.487638 (+ 8514us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.487654 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.487683 (+ 29us) server_negotiation.cc:407] Connection header received
0809 07:48:49.487776 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.487784 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.488036 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.488234 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:49.488554 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.489690 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.493830 (+ 4140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.494653 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.494755 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.494900 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.499198 (+ 4298us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.499350 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8359,"thread_start_us":113,"threads_started":1}
I20260809 07:48:49.499992 26531 tablet_service.cc:1467] Processing CreateTablet for tablet 8c43c33dfa6f4bacaf565681b874706e (DEFAULT_TABLE table=test-table [id=87aee70b0e0e4b258b58697efaf793e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:49.501106 26235 tablet_service.cc:1467] Processing CreateTablet for tablet 8c43c33dfa6f4bacaf565681b874706e (DEFAULT_TABLE table=test-table [id=87aee70b0e0e4b258b58697efaf793e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:49.502039 26531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c43c33dfa6f4bacaf565681b874706e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:49.502094 26235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c43c33dfa6f4bacaf565681b874706e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:49.502842 26383 tablet_service.cc:1467] Processing CreateTablet for tablet 8c43c33dfa6f4bacaf565681b874706e (DEFAULT_TABLE table=test-table [id=87aee70b0e0e4b258b58697efaf793e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:49.504083 26383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c43c33dfa6f4bacaf565681b874706e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:49.511656 26640 tablet_bootstrap.cc:492] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Bootstrap starting.
I20260809 07:48:49.513628 26640 tablet_bootstrap.cc:654] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:49.514380 26640 log.cc:824] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:49.515730 26641 tablet_bootstrap.cc:492] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Bootstrap starting.
I20260809 07:48:49.516491 26640 tablet_bootstrap.cc:492] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: No bootstrap required, opened a new log
I20260809 07:48:49.516698 26640 ts_tablet_manager.cc:1397] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 07:48:49.516817 26639 tablet_bootstrap.cc:492] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Bootstrap starting.
I20260809 07:48:49.518532 26641 tablet_bootstrap.cc:654] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:49.519090 26639 tablet_bootstrap.cc:654] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:49.519464 26641 log.cc:824] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:49.519969 26639 log.cc:824] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:49.519990 26640 raft_consensus.cc:348] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.520308 26640 raft_consensus.cc:374] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:49.520375 26640 raft_consensus.cc:729] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fdb83dc40c4a308ef7a8b27bdae12e, State: Initialized, Role: FOLLOWER
I20260809 07:48:49.520911 26640 consensus_queue.cc:260] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.521803 26641 tablet_bootstrap.cc:492] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: No bootstrap required, opened a new log
I20260809 07:48:49.522019 26641 ts_tablet_manager.cc:1397] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:48:49.521962 26640 ts_tablet_manager.cc:1428] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 07:48:49.522495 26307 heartbeater.cc:503] Master 127.18.71.126:39719 was elected leader, sending a full tablet report...
I20260809 07:48:49.526794 26641 raft_consensus.cc:348] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.526997 26641 raft_consensus.cc:374] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:49.527027 26641 raft_consensus.cc:729] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fec0adaae5d4abd836e0233375d59a1, State: Initialized, Role: FOLLOWER
I20260809 07:48:49.527443 26641 consensus_queue.cc:260] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.528647 26641 ts_tablet_manager.cc:1428] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 07:48:49.529260 26453 heartbeater.cc:503] Master 127.18.71.126:39719 was elected leader, sending a full tablet report...
I20260809 07:48:49.530146 26639 tablet_bootstrap.cc:492] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: No bootstrap required, opened a new log
I20260809 07:48:49.530383 26639 ts_tablet_manager.cc:1397] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260809 07:48:49.534996 26639 raft_consensus.cc:348] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.535231 26639 raft_consensus.cc:374] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:49.535292 26639 raft_consensus.cc:729] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00cd805fbc448f8acedd458bc71d24f, State: Initialized, Role: FOLLOWER
I20260809 07:48:49.535707 26639 consensus_queue.cc:260] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.536350 26601 heartbeater.cc:503] Master 127.18.71.126:39719 was elected leader, sending a full tablet report...
I20260809 07:48:49.536772 26639 ts_tablet_manager.cc:1428] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 07:48:49.668949 26604 tablet.cc:2369] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:48:49.685194 26457 tablet.cc:2369] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:49.703151 26646 raft_consensus.cc:482] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:49.703419 26646 raft_consensus.cc:504] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.704766 26646 leader_election.cc:288] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023), 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833)
I20260809 07:48:49.708778 26633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.705113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:37435 (local address 127.18.71.67:33023)
0809 07:48:49.705256 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.705261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.705464 (+ 203us) server_negotiation.cc:407] Connection header received
0809 07:48:49.705640 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.705643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.705687 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.705769 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:49.706085 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.706571 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.707711 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.708113 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.708238 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.708297 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.708577 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.708661 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:48:49.708775 26637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.705280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:53875 (local address 127.18.71.65:37833)
0809 07:48:49.705421 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.705429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.705466 (+ 37us) server_negotiation.cc:407] Connection header received
0809 07:48:49.705643 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.705646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.705694 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.705772 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:49.706064 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.706588 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.707708 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.708112 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.708238 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.708296 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.708556 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.708634 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:48:49.708776 26648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.705081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37833 (local address 127.18.71.66:53875)
0809 07:48:49.705389 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.705404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.705531 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.705785 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.705788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.705824 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:49.705949 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.705954 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.706704 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.706708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.707583 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.707592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.708238 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.708242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.708404 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.708423 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.708539 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.708620 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":76,"spinlock_wait_cycles":5504,"thread_start_us":72,"threads_started":1}
I20260809 07:48:49.708791 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.704998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33023 (local address 127.18.71.66:37435)
0809 07:48:49.705357 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:49.705377 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:49.705508 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:49.705784 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:49.705787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:49.705814 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:49.705969 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.705974 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.706689 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.706694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.707581 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:49.707590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.708238 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.708242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:49.708430 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.708466 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 07:48:49.708557 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 07:48:49.708636 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":115,"spinlock_wait_cycles":1920,"thread_start_us":96,"threads_started":1}
I20260809 07:48:49.709741 26554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c43c33dfa6f4bacaf565681b874706e" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00cd805fbc448f8acedd458bc71d24f" is_pre_election: true
I20260809 07:48:49.709745 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c43c33dfa6f4bacaf565681b874706e" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" is_pre_election: true
I20260809 07:48:49.710076 26257 raft_consensus.cc:2454] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 0.
I20260809 07:48:49.710076 26554 raft_consensus.cc:2454] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 0.
I20260809 07:48:49.710533 26334 leader_election.cc:302] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fec0adaae5d4abd836e0233375d59a1, f00cd805fbc448f8acedd458bc71d24f; no voters:
I20260809 07:48:49.710800 26646 raft_consensus.cc:2781] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:49.710893 26646 raft_consensus.cc:482] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:49.710945 26646 raft_consensus.cc:3037] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:49.711982 26646 raft_consensus.cc:504] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.712334 26646 leader_election.cc:288] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 election: Requested vote from peers f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023), 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833)
I20260809 07:48:49.712731 26554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c43c33dfa6f4bacaf565681b874706e" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00cd805fbc448f8acedd458bc71d24f"
I20260809 07:48:49.712852 26554 raft_consensus.cc:3037] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:49.713867 26554 raft_consensus.cc:2454] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 1.
I20260809 07:48:49.714154 26334 leader_election.cc:302] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9fec0adaae5d4abd836e0233375d59a1, f00cd805fbc448f8acedd458bc71d24f; no voters:
I20260809 07:48:49.714145 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c43c33dfa6f4bacaf565681b874706e" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e"
I20260809 07:48:49.714291 26257 raft_consensus.cc:3037] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:49.714331 26646 raft_consensus.cc:2781] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:49.714548 26646 raft_consensus.cc:686] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 1 LEADER]: Becoming Leader. State: Replica: 9fec0adaae5d4abd836e0233375d59a1, State: Running, Role: LEADER
I20260809 07:48:49.714879 26646 consensus_queue.cc:237] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } }
I20260809 07:48:49.715433 26257 raft_consensus.cc:2454] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 1.
I20260809 07:48:49.718612 25284 catalog_manager.cc:5495] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fec0adaae5d4abd836e0233375d59a1 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "9fec0adaae5d4abd836e0233375d59a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } health_report { overall_health: UNKNOWN } } }
W20260809 07:48:49.727707 26311 tablet.cc:2369] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:49.735658 26635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.732164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.18.71.66:35097)
0809 07:48:49.732288 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.732291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.732304 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:48:49.732436 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.732439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.732558 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.732653 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:49.732877 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.733369 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.734181 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.734389 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.734469 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.734783 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:49.735200 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:49.735289 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.735412 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.735464 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:48:49.741859 26554 raft_consensus.cc:1261] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Refusing update from remote peer 9fec0adaae5d4abd836e0233375d59a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:49.741858 26257 raft_consensus.cc:1261] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 1 FOLLOWER]: Refusing update from remote peer 9fec0adaae5d4abd836e0233375d59a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:49.742619 26650 consensus_queue.cc:1035] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:49.742797 26646 consensus_queue.cc:1035] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:49.765229 25284 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:49.765439 25284 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53270:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:49.770011 25284 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e806a0baba384622928f82daa2c3da62: Sending DeleteTablet for 3 replicas of tablet 8c43c33dfa6f4bacaf565681b874706e
I20260809 07:48:49.770941 26383 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c43c33dfa6f4bacaf565681b874706e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:49 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:60476
I20260809 07:48:49.771147 26531 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c43c33dfa6f4bacaf565681b874706e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:49 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:51428
I20260809 07:48:49.771540 26235 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c43c33dfa6f4bacaf565681b874706e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:49 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:38788
I20260809 07:48:49.771692 26665 tablet_replica.cc:330] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: stopping tablet replica
I20260809 07:48:49.772058 26664 tablet_replica.cc:330] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: stopping tablet replica
I20260809 07:48:49.772073 26665 raft_consensus.cc:2229] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:49.772375 26664 raft_consensus.cc:2229] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:49.772492 26665 raft_consensus.cc:2258] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:49.772854 26664 raft_consensus.cc:2258] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:49.774200 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 25249
I20260809 07:48:49.776837 26664 ts_tablet_manager.cc:1905] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:49.776976 26665 ts_tablet_manager.cc:1905] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:49.779731 26664 ts_tablet_manager.cc:1918] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:49.779824 26664 log.cc:1196] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/8c43c33dfa6f4bacaf565681b874706e
I20260809 07:48:49.780185 26664 ts_tablet_manager.cc:1939] T 8c43c33dfa6f4bacaf565681b874706e P 9fec0adaae5d4abd836e0233375d59a1: Deleting consensus metadata
I20260809 07:48:49.783095 26665 ts_tablet_manager.cc:1918] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:49.783231 26665 log.cc:1196] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/8c43c33dfa6f4bacaf565681b874706e
I20260809 07:48:49.783694 26665 ts_tablet_manager.cc:1939] T 8c43c33dfa6f4bacaf565681b874706e P f00cd805fbc448f8acedd458bc71d24f: Deleting consensus metadata
W20260809 07:48:49.786211 26333 connection.cc:535] server connection from 127.0.0.1:60476 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:48:49.786281 26484 connection.cc:535] server connection from 127.0.0.1:51428 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:48:49.786978 26666 tablet_replica.cc:330] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: stopping tablet replica
I20260809 07:48:49.787402 26666 raft_consensus.cc:2229] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:49.787792 26666 raft_consensus.cc:2258] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:49.789858 26666 ts_tablet_manager.cc:1905] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:49.792979 26666 ts_tablet_manager.cc:1918] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:49.793169 26666 log.cc:1196] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/8c43c33dfa6f4bacaf565681b874706e
I20260809 07:48:49.793601 26666 ts_tablet_manager.cc:1939] T 8c43c33dfa6f4bacaf565681b874706e P 29fdb83dc40c4a308ef7a8b27bdae12e: Deleting consensus metadata
W20260809 07:48:49.794907 26186 connection.cc:441] server connection from 127.0.0.1:38788 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:38788 (request call id 1) could send its response
I20260809 07:48:49.796257 26616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.772754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.18.71.125:44245)
0809 07:48:49.772885 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.772892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.781060 (+ 8168us) server_negotiation.cc:407] Connection header received
0809 07:48:49.781291 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.781298 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.781373 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.781440 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.785213 (+ 3773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.786116 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.787959 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.788339 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.788436 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.791101 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.791105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.791122 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.791155 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.793229 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.793917 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.793923 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.793926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.794028 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.794396 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.794399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.794401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.794655 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.794775 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.795963 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.796095 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 07:48:49.803504 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.791483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.18.71.125:44245)
0809 07:48:49.791636 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.791641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.792440 (+ 799us) server_negotiation.cc:407] Connection header received
0809 07:48:49.792698 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.792704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.792762 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.792816 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.794592 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.795430 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.796886 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.797222 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.797298 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.799660 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.799663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.799666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.799690 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.801454 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.802010 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.802014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.802015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.802109 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.802452 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.802457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.802460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.802703 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.802815 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.803182 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.803301 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:48:49.805030 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.773049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54090 (local address 127.18.71.124:38191)
0809 07:48:49.773178 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.773183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.795565 (+ 22382us) server_negotiation.cc:407] Connection header received
0809 07:48:49.795624 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.795627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.795702 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.795804 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.797015 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.797820 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.798694 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.799045 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.799164 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.800861 (+ 1697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.800867 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.800873 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.800904 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.802698 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.803203 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.803206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.803208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.803294 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.803603 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.803606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.803608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.803822 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.804671 (+ 849us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.804750 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.804881 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 07:48:49.810109 26312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:49.801324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54098 (local address 127.18.71.124:38191)
0809 07:48:49.801432 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:49.801437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:49.801616 (+ 179us) server_negotiation.cc:407] Connection header received
0809 07:48:49.801789 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:49.801792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:49.801836 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:49.801883 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:49.802700 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.803464 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.804992 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:49.805340 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:49.805413 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:49.807124 (+ 1711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:49.807127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:49.807129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:49.807150 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:49.808620 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.809068 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.809071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.809072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.809135 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:49.809388 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:49.809390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:49.809391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:49.809542 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:49.809637 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:49.809867 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 07:48:49.809955 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 07:48:50.756923 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:50.756279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:46653)
0809 07:48:50.756692 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:50.756797 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":86,"threads_started":1}
I20260809 07:48:50.757418 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:50.756327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:36871)
0809 07:48:50.757215 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:50.757317 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":75,"threads_started":1}
W20260809 07:48:50.757958 26601 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:39719 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
W20260809 07:48:50.758248 26307 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:39719 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
I20260809 07:48:50.758873 26676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:50.758376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:55065)
0809 07:48:50.758725 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:50.758798 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":116,"threads_started":1}
W20260809 07:48:50.759261 26453 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:39719 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
I20260809 07:48:51.270217 26678 raft_consensus.cc:482] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e806a0baba384622928f82daa2c3da62)
I20260809 07:48:51.270498 26678 raft_consensus.cc:504] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:51.270637 26679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e806a0baba384622928f82daa2c3da62)
I20260809 07:48:51.270772 26679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:51.271360 26679 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719), e850e16be7ad411bbc4a4b035272faee (127.18.71.124:38191)
I20260809 07:48:51.271727 26678 leader_election.cc:288] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719), 2d2a5e802b2046228b385fbff6840a0e (127.18.71.125:44245)
I20260809 07:48:51.271862 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.271465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53780)
0809 07:48:51.271660 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.271752 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260809 07:48:51.272305 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.271806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53790)
0809 07:48:51.272104 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.272177 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":71,"threads_started":1}
W20260809 07:48:51.272491 25903 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
W20260809 07:48:51.272584 26097 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
I20260809 07:48:51.273622 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.273384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53814)
0809 07:48:51.273509 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.273559 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 07:48:51.273622 26682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.273357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53802)
0809 07:48:51.273512 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.273559 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260809 07:48:51.273761 26097 leader_election.cc:334] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719): Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
W20260809 07:48:51.273800 25903 leader_election.cc:334] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719): Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
I20260809 07:48:51.276885 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.271790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:38191 (local address 127.0.0.1:54102)
0809 07:48:51.272033 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.272049 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:51.272150 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:51.272828 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:51.272832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:51.272853 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:51.273000 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:51.273007 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.274160 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.274162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:51.275501 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:51.275514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.276414 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.276418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:51.276545 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:51.276574 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 07:48:51.276660 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 07:48:51.276756 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":71,"threads_started":1}
I20260809 07:48:51.276964 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.272186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.18.71.124:38191)
0809 07:48:51.272571 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:51.272577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:51.272591 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:48:51.272639 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:51.272641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:51.272689 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:51.273122 (+ 433us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:51.273220 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.274042 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.275639 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.276287 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.276396 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:51.276561 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:51.276698 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 07:48:51.276829 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":320,"thread_start_us":88,"threads_started":1}
I20260809 07:48:51.276983 26685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.272314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48252 (local address 127.18.71.125:44245)
0809 07:48:51.272576 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:51.272581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:51.272823 (+ 242us) server_negotiation.cc:407] Connection header received
0809 07:48:51.273013 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:51.273015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:51.273060 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:51.273135 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:51.273417 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.274174 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.275639 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.276287 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.276396 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:51.276500 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:51.276735 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 07:48:51.276821 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":202,"thread_start_us":57,"threads_started":1}
I20260809 07:48:51.276996 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.272315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:44245 (local address 127.0.0.1:48252)
0809 07:48:51.272737 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.272751 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:51.272896 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:51.273220 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:51.273223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:51.273237 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:51.273304 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:51.273309 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.274276 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.274279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:51.275506 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:51.275514 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:51.276414 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:51.276418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:51.276598 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:51.276633 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 07:48:51.276708 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 07:48:51.276833 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":44,"thread_start_us":45,"threads_started":1}
I20260809 07:48:51.277438 25928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e850e16be7ad411bbc4a4b035272faee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d2a5e802b2046228b385fbff6840a0e" is_pre_election: true
I20260809 07:48:51.277530 26128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d2a5e802b2046228b385fbff6840a0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e850e16be7ad411bbc4a4b035272faee" is_pre_election: true
I20260809 07:48:51.277676 25928 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e850e16be7ad411bbc4a4b035272faee in term 1.
I20260809 07:48:51.277729 26128 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2a5e802b2046228b385fbff6840a0e in term 1.
I20260809 07:48:51.278074 25903 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d2a5e802b2046228b385fbff6840a0e, e850e16be7ad411bbc4a4b035272faee; no voters: e806a0baba384622928f82daa2c3da62
I20260809 07:48:51.278053 26094 leader_election.cc:302] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d2a5e802b2046228b385fbff6840a0e, e850e16be7ad411bbc4a4b035272faee; no voters: e806a0baba384622928f82daa2c3da62
I20260809 07:48:51.278264 26679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:48:51.278329 26679 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Starting leader election (detected failure of leader e806a0baba384622928f82daa2c3da62)
I20260809 07:48:51.278358 26679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:51.278436 26678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 07:48:51.278523 26678 raft_consensus.cc:482] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Starting leader election (detected failure of leader e806a0baba384622928f82daa2c3da62)
I20260809 07:48:51.278577 26678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:48:51.279508 26679 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:51.279541 26678 raft_consensus.cc:504] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:51.279846 26679 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 2 election: Requested vote from peers e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719), e850e16be7ad411bbc4a4b035272faee (127.18.71.124:38191)
I20260809 07:48:51.279978 26678 leader_election.cc:288] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [CANDIDATE]: Term 2 election: Requested vote from peers e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719), 2d2a5e802b2046228b385fbff6840a0e (127.18.71.125:44245)
I20260809 07:48:51.280282 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.280069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53832)
0809 07:48:51.280171 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.280224 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 07:48:51.280323 25928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e850e16be7ad411bbc4a4b035272faee" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2d2a5e802b2046228b385fbff6840a0e"
I20260809 07:48:51.280371 26128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d2a5e802b2046228b385fbff6840a0e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e850e16be7ad411bbc4a4b035272faee"
I20260809 07:48:51.280526 25928 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e850e16be7ad411bbc4a4b035272faee in current term 2: Already voted for candidate 2d2a5e802b2046228b385fbff6840a0e in this term.
I20260809 07:48:51.280540 26128 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d2a5e802b2046228b385fbff6840a0e in current term 2: Already voted for candidate e850e16be7ad411bbc4a4b035272faee in this term.
I20260809 07:48:51.280546 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.280000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53820)
0809 07:48:51.280248 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.280304 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185}
I20260809 07:48:51.281306 26681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.281084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53838)
0809 07:48:51.281196 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.281248 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 07:48:51.281395 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.281174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:53850)
0809 07:48:51.281278 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.281322 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 07:48:51.281421 25903 leader_election.cc:334] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719): Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
W20260809 07:48:51.281492 26097 leader_election.cc:334] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719): Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
I20260809 07:48:51.281531 25903 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2d2a5e802b2046228b385fbff6840a0e; no voters: e806a0baba384622928f82daa2c3da62, e850e16be7ad411bbc4a4b035272faee
I20260809 07:48:51.281563 26097 leader_election.cc:302] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e850e16be7ad411bbc4a4b035272faee; no voters: 2d2a5e802b2046228b385fbff6840a0e, e806a0baba384622928f82daa2c3da62
I20260809 07:48:51.281668 26679 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:48:51.281798 26678 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 07:48:51.759485 26688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.758805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:34363)
0809 07:48:51.759297 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.759372 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":109,"threads_started":1}
I20260809 07:48:51.759778 26689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.758800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:48519)
0809 07:48:51.759606 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.759676 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":117,"threads_started":1}
I20260809 07:48:51.760340 26690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:51.759987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:51535)
0809 07:48:51.760216 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:51.760266 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":83,"threads_started":1}
I20260809 07:48:52.761046 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:52.760553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:53013)
0809 07:48:52.760849 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:52.760927 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":91,"threads_started":1}
I20260809 07:48:52.761142 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:52.760504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:39627)
0809 07:48:52.760970 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:52.761037 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":82,"threads_started":1}
I20260809 07:48:52.761994 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:52.761654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:48525)
0809 07:48:52.761870 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:52.761920 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":64,"threads_started":1}
I20260809 07:48:53.161959 26694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:53.162110 26694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:53.162617 26694 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719), e850e16be7ad411bbc4a4b035272faee (127.18.71.124:38191)
I20260809 07:48:53.163285 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.162875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:54116)
0809 07:48:53.163102 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.163180 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260809 07:48:53.163511 26128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d2a5e802b2046228b385fbff6840a0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e850e16be7ad411bbc4a4b035272faee" is_pre_election: true
I20260809 07:48:53.163707 26128 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d2a5e802b2046228b385fbff6840a0e in term 2.
I20260809 07:48:53.164134 25903 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d2a5e802b2046228b385fbff6840a0e, e850e16be7ad411bbc4a4b035272faee; no voters:
I20260809 07:48:53.164237 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.164001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:54126)
0809 07:48:53.164130 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.164181 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 07:48:53.164341 26694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 07:48:53.164422 25903 leader_election.cc:334] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719): Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
I20260809 07:48:53.164434 26694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:53.164551 26694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:53.165653 26694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:53.165983 26694 leader_election.cc:288] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 3 election: Requested vote from peers e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719), e850e16be7ad411bbc4a4b035272faee (127.18.71.124:38191)
I20260809 07:48:53.166337 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.166098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:54128)
0809 07:48:53.166200 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.166272 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 07:48:53.166519 26128 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d2a5e802b2046228b385fbff6840a0e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e850e16be7ad411bbc4a4b035272faee"
I20260809 07:48:53.166673 26128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:48:53.167047 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.166849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:54138)
0809 07:48:53.166944 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.166995 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 07:48:53.167217 25903 leader_election.cc:334] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719): Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
I20260809 07:48:53.167824 26128 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d2a5e802b2046228b385fbff6840a0e in term 3.
I20260809 07:48:53.168112 25903 leader_election.cc:302] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d2a5e802b2046228b385fbff6840a0e, e850e16be7ad411bbc4a4b035272faee; no voters: e806a0baba384622928f82daa2c3da62
I20260809 07:48:53.168321 26694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:48:53.168573 26694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [term 3 LEADER]: Becoming Leader. State: Replica: 2d2a5e802b2046228b385fbff6840a0e, State: Running, Role: LEADER
I20260809 07:48:53.168876 26694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } }
I20260809 07:48:53.170024 26698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2a5e802b2046228b385fbff6840a0e. Latest consensus state: current_term: 3 leader_uuid: "2d2a5e802b2046228b385fbff6840a0e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:53.170158 26698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:53.170503 26700 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:53.173269 26700 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:53.173710 26700 catalog_manager.cc:1259] Loaded cluster ID: 96f4bd9805ba4a43bf10343a6b58a3bc
I20260809 07:48:53.173787 26700 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:53.174407 26700 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:53.174835 26700 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Loaded TSK: 0
I20260809 07:48:53.175592 26700 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:48:53.199975 25916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:53.203275 25916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:53.206683 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.206408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:54160)
0809 07:48:53.206557 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.206614 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 07:48:53.206691 26128 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [term 3 FOLLOWER]: Refusing update from remote peer 2d2a5e802b2046228b385fbff6840a0e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 07:48:53.207252 26698 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 07:48:53.207640 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.207372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.0.0.1:54170)
0809 07:48:53.207524 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.207584 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260809 07:48:53.207823 25903 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e -> Peer e806a0baba384622928f82daa2c3da62 (127.18.71.126:39719): Couldn't send request to peer e806a0baba384622928f82daa2c3da62. Status: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:48:53.208982 26702 sys_catalog.cc:455] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d2a5e802b2046228b385fbff6840a0e. Latest consensus state: current_term: 3 leader_uuid: "2d2a5e802b2046228b385fbff6840a0e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:53.209149 26702 sys_catalog.cc:458] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:53.210357 26694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2d2a5e802b2046228b385fbff6840a0e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:53.210505 26694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:53.210855 26702 sys_catalog.cc:455] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2d2a5e802b2046228b385fbff6840a0e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "e806a0baba384622928f82daa2c3da62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 39719 } } peers { permanent_uuid: "2d2a5e802b2046228b385fbff6840a0e" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 44245 } attrs { promote: false } } peers { permanent_uuid: "e850e16be7ad411bbc4a4b035272faee" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 38191 } attrs { promote: false } } }
I20260809 07:48:53.211154 26702 sys_catalog.cc:458] T 00000000000000000000000000000000 P e850e16be7ad411bbc4a4b035272faee [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:48:53.222616 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.217655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35097 (local address 127.0.0.1:60488)
0809 07:48:53.218223 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.218239 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:53.218350 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:53.218720 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:53.218723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:53.218740 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:53.218859 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:53.218865 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.220036 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.220042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:53.221334 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:53.221344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.222143 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.222147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:53.222249 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:53.222279 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 07:48:53.222371 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 07:48:53.222446 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":1,"spinlock_wait_cycles":6784,"thread_start_us":101,"threads_started":1}
I20260809 07:48:53.222645 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.218065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60488 (local address 127.18.71.66:35097)
0809 07:48:53.218347 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:53.218352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:53.218367 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:53.218443 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:53.218447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:53.218509 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:53.218599 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:53.219024 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.219894 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.221502 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.222012 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.222125 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:53.222192 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:53.222399 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 07:48:53.222494 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":119,"threads_started":1}
I20260809 07:48:53.222971 26695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.217036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37833 (local address 127.0.0.1:38790)
0809 07:48:53.217144 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.217163 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:53.217262 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:53.218301 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:53.218304 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:53.218340 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:53.218467 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:53.218474 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.219781 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.219785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:53.221512 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:53.221524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.222539 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.222543 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:53.222650 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:53.222685 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 07:48:53.222772 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 07:48:53.222857 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":363,"spinlock_wait_cycles":9856}
I20260809 07:48:53.223127 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.217985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33023 (local address 127.0.0.1:51432)
0809 07:48:53.218742 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.218761 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:53.218872 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:53.219136 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:53.219139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:53.219156 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:53.219274 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:53.219280 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.220349 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.220356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:53.221734 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:53.221743 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.222704 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.222708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:53.222811 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:53.222839 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 07:48:53.222915 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 07:48:53.222996 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":41,"thread_start_us":102,"threads_started":1}
I20260809 07:48:53.223351 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.217187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.18.71.65:37833)
0809 07:48:53.217725 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:53.217730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:53.217746 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:48:53.217807 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:53.217812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:53.217864 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:53.217980 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:53.218626 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.219640 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.221668 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.222290 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.222397 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:53.222475 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:53.223059 (+ 584us) server_negotiation.cc:299] Negotiation successful
0809 07:48:53.223165 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":99,"threads_started":1}
I20260809 07:48:53.223408 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.218378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.18.71.67:33023)
0809 07:48:53.218666 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:53.218671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:53.218837 (+ 166us) server_negotiation.cc:407] Connection header received
0809 07:48:53.218967 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:53.218970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:53.219024 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:53.219126 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:53.219403 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.220225 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.221858 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.222476 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.222579 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:53.222638 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:53.222936 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 07:48:53.223014 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":123,"threads_started":1}
I20260809 07:48:53.223937 26531 tablet_service.cc:1467] Processing CreateTablet for tablet 8d78f49b409c4d7b9136830b9b2842f7 (DEFAULT_TABLE table=test-table [id=290b69bcbd16437db3aa57038dffe5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:53.224143 26235 tablet_service.cc:1467] Processing CreateTablet for tablet 8d78f49b409c4d7b9136830b9b2842f7 (DEFAULT_TABLE table=test-table [id=290b69bcbd16437db3aa57038dffe5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:53.224289 26531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d78f49b409c4d7b9136830b9b2842f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:53.224255 26383 tablet_service.cc:1467] Processing CreateTablet for tablet 8d78f49b409c4d7b9136830b9b2842f7 (DEFAULT_TABLE table=test-table [id=290b69bcbd16437db3aa57038dffe5ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:53.224381 26235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d78f49b409c4d7b9136830b9b2842f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:53.224557 26383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d78f49b409c4d7b9136830b9b2842f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:53.226686 26714 tablet_bootstrap.cc:492] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e: Bootstrap starting.
I20260809 07:48:53.227408 26715 tablet_bootstrap.cc:492] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: Bootstrap starting.
I20260809 07:48:53.227738 26714 tablet_bootstrap.cc:654] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:53.228464 26715 tablet_bootstrap.cc:654] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:53.229156 26716 tablet_bootstrap.cc:492] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: Bootstrap starting.
I20260809 07:48:53.229395 26714 tablet_bootstrap.cc:492] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e: No bootstrap required, opened a new log
I20260809 07:48:53.229516 26714 ts_tablet_manager.cc:1397] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:53.230080 26716 tablet_bootstrap.cc:654] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:53.230134 26715 tablet_bootstrap.cc:492] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: No bootstrap required, opened a new log
I20260809 07:48:53.230119 26714 raft_consensus.cc:348] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.230278 26715 ts_tablet_manager.cc:1397] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:53.230276 26714 raft_consensus.cc:374] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:53.230324 26714 raft_consensus.cc:729] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29fdb83dc40c4a308ef7a8b27bdae12e, State: Initialized, Role: FOLLOWER
I20260809 07:48:53.230497 26714 consensus_queue.cc:260] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.230937 26714 ts_tablet_manager.cc:1428] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 07:48:53.230994 26715 raft_consensus.cc:348] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.231112 26715 raft_consensus.cc:374] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:53.231142 26715 raft_consensus.cc:729] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f00cd805fbc448f8acedd458bc71d24f, State: Initialized, Role: FOLLOWER
I20260809 07:48:53.231307 26715 consensus_queue.cc:260] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.231807 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.231405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:37305)
0809 07:48:53.231665 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.231726 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 07:48:53.231876 26715 ts_tablet_manager.cc:1428] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:53.232596 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.232224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:56047)
0809 07:48:53.232429 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.232525 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260809 07:48:53.233474 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.233160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:38085)
0809 07:48:53.233319 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.233378 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 07:48:53.234051 26716 tablet_bootstrap.cc:492] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: No bootstrap required, opened a new log
I20260809 07:48:53.234591 26716 ts_tablet_manager.cc:1397] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 07:48:53.234835 26603 heartbeater.cc:503] Master 127.18.71.125:44245 was elected leader, sending a full tablet report...
I20260809 07:48:53.235149 26716 raft_consensus.cc:348] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.235292 26716 raft_consensus.cc:374] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:53.235344 26716 raft_consensus.cc:729] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fec0adaae5d4abd836e0233375d59a1, State: Initialized, Role: FOLLOWER
I20260809 07:48:53.236079 26716 consensus_queue.cc:260] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.236539 26716 ts_tablet_manager.cc:1428] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:53.237255 26310 heartbeater.cc:503] Master 127.18.71.125:44245 was elected leader, sending a full tablet report...
I20260809 07:48:53.237429 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.236998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:53413)
0809 07:48:53.237305 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.237361 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260809 07:48:53.238145 26456 heartbeater.cc:503] Master 127.18.71.125:44245 was elected leader, sending a full tablet report...
I20260809 07:48:53.238323 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.238081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:32945)
0809 07:48:53.238223 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.238270 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 07:48:53.239015 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.237959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:57507)
0809 07:48:53.238803 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.238862 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":751}
I20260809 07:48:53.241286 26722 raft_consensus.cc:482] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:53.241380 26722 raft_consensus.cc:504] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.241775 26722 leader_election.cc:288] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833), f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023)
I20260809 07:48:53.242153 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d78f49b409c4d7b9136830b9b2842f7" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" is_pre_election: true
I20260809 07:48:53.242311 26257 raft_consensus.cc:2454] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 0.
I20260809 07:48:53.242287 26554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d78f49b409c4d7b9136830b9b2842f7" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00cd805fbc448f8acedd458bc71d24f" is_pre_election: true
I20260809 07:48:53.242441 26554 raft_consensus.cc:2454] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 0.
I20260809 07:48:53.242633 26336 leader_election.cc:302] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fdb83dc40c4a308ef7a8b27bdae12e, 9fec0adaae5d4abd836e0233375d59a1; no voters:
I20260809 07:48:53.242794 26722 raft_consensus.cc:2781] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:53.242856 26722 raft_consensus.cc:482] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:53.242897 26722 raft_consensus.cc:3037] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:53.243958 26722 raft_consensus.cc:504] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.244318 26722 leader_election.cc:288] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 election: Requested vote from peers 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833), f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023)
I20260809 07:48:53.244623 26257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d78f49b409c4d7b9136830b9b2842f7" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e"
I20260809 07:48:53.244693 26554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d78f49b409c4d7b9136830b9b2842f7" candidate_uuid: "9fec0adaae5d4abd836e0233375d59a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f00cd805fbc448f8acedd458bc71d24f"
I20260809 07:48:53.244751 26257 raft_consensus.cc:3037] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:53.244814 26554 raft_consensus.cc:3037] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:53.245640 26257 raft_consensus.cc:2454] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 1.
I20260809 07:48:53.245797 26554 raft_consensus.cc:2454] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fec0adaae5d4abd836e0233375d59a1 in term 1.
I20260809 07:48:53.245908 26336 leader_election.cc:302] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29fdb83dc40c4a308ef7a8b27bdae12e, 9fec0adaae5d4abd836e0233375d59a1; no voters:
I20260809 07:48:53.246088 26722 raft_consensus.cc:2781] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:53.246294 26722 raft_consensus.cc:686] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 1 LEADER]: Becoming Leader. State: Replica: 9fec0adaae5d4abd836e0233375d59a1, State: Running, Role: LEADER
I20260809 07:48:53.246435 26722 consensus_queue.cc:237] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } }
I20260809 07:48:53.247136 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.246851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:47151)
0809 07:48:53.247009 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.247073 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 07:48:53.247961 25914 catalog_manager.cc:5495] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fec0adaae5d4abd836e0233375d59a1 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "9fec0adaae5d4abd836e0233375d59a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fec0adaae5d4abd836e0233375d59a1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35097 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 } health_report { overall_health: UNKNOWN } } }
I20260809 07:48:53.262940 26711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.259629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.18.71.66:35097)
0809 07:48:53.259761 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:53.259768 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:53.259786 (+ 18us) server_negotiation.cc:407] Connection header received
0809 07:48:53.259958 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:53.259963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:53.260016 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:53.260099 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:53.260392 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.260971 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.261829 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:53.262053 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:53.262162 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:53.262415 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:53.262494 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:53.262573 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:53.262736 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 07:48:53.262793 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:48:53.265465 26554 raft_consensus.cc:1261] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Refusing update from remote peer 9fec0adaae5d4abd836e0233375d59a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:53.265506 26257 raft_consensus.cc:1261] T 8d78f49b409c4d7b9136830b9b2842f7 P 29fdb83dc40c4a308ef7a8b27bdae12e [term 1 FOLLOWER]: Refusing update from remote peer 9fec0adaae5d4abd836e0233375d59a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:53.265933 26723 consensus_queue.cc:1035] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f00cd805fbc448f8acedd458bc71d24f" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:53.266117 26722 consensus_queue.cc:1035] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29fdb83dc40c4a308ef7a8b27bdae12e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:53.267690 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.267322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:46259)
0809 07:48:53.267512 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.267590 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 07:48:53.267922 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.267632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:45595)
0809 07:48:53.267785 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.267837 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 07:48:53.271318 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.271057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:53641)
0809 07:48:53.271199 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.271251 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 07:48:53.272006 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.271787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.66:35925)
0809 07:48:53.271896 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.271950 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 07:48:53.272940 26692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.272586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.65:41471)
0809 07:48:53.272797 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.272860 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260809 07:48:53.273383 26728 mvcc.cc:205] Tried to move back new op lower bound from 7316528059450257408 to 7316528059378847744. Current Snapshot: MvccSnapshot[applied={T|T < 7316528059450257408}]
I20260809 07:48:53.274732 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.274381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:39719 (local address 127.18.71.67:58955)
0809 07:48:53.274596 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.274653 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140}
I20260809 07:48:53.275753 26727 mvcc.cc:205] Tried to move back new op lower bound from 7316528059450257408 to 7316528059378847744. Current Snapshot: MvccSnapshot[applied={T|T < 7316528059450257408 or (T in {7316528059450257408})}]
I20260809 07:48:53.275846 25914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:53.275997 25914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:53.280251 25914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d2a5e802b2046228b385fbff6840a0e: Sending DeleteTablet for 3 replicas of tablet 8d78f49b409c4d7b9136830b9b2842f7
I20260809 07:48:53.280874 26531 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d78f49b409c4d7b9136830b9b2842f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:53 UTC) from {username='oryx', principal='oryx/127.18.71.125@KRBTEST.COM'} at 127.0.0.1:51432
I20260809 07:48:53.281035 26530 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d78f49b409c4d7b9136830b9b2842f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:53 UTC) from {username='oryx', principal='oryx/127.18.71.125@KRBTEST.COM'} at 127.0.0.1:51432
I20260809 07:48:53.281036 26383 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d78f49b409c4d7b9136830b9b2842f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:53 UTC) from {username='oryx', principal='oryx/127.18.71.125@KRBTEST.COM'} at 127.0.0.1:60488
I20260809 07:48:53.281380 26736 tablet_replica.cc:330] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: stopping tablet replica
I20260809 07:48:53.281399 26735 tablet_replica.cc:330] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: stopping tablet replica
I20260809 07:48:53.281519 26736 raft_consensus.cc:2229] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:53.281533 26735 raft_consensus.cc:2229] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:53.281666 26735 raft_consensus.cc:2258] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:53.281713 26736 raft_consensus.cc:2258] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:53.281985 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 26071
I20260809 07:48:53.284581 26736 ts_tablet_manager.cc:1905] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:53.284986 26735 ts_tablet_manager.cc:1905] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:53.286947 26736 ts_tablet_manager.cc:1918] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:53.287048 26736 log.cc:1196] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/8d78f49b409c4d7b9136830b9b2842f7
I20260809 07:48:53.287410 26736 ts_tablet_manager.cc:1939] T 8d78f49b409c4d7b9136830b9b2842f7 P 9fec0adaae5d4abd836e0233375d59a1: Deleting consensus metadata
I20260809 07:48:53.288111 25900 catalog_manager.cc:4841] TS 9fec0adaae5d4abd836e0233375d59a1 (127.18.71.66:35097): tablet 8d78f49b409c4d7b9136830b9b2842f7 (table test-table [id=290b69bcbd16437db3aa57038dffe5ed]) successfully deleted
I20260809 07:48:53.288168 26735 ts_tablet_manager.cc:1918] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:53.288252 26735 log.cc:1196] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/8d78f49b409c4d7b9136830b9b2842f7
I20260809 07:48:53.288580 26735 ts_tablet_manager.cc:1939] T 8d78f49b409c4d7b9136830b9b2842f7 P f00cd805fbc448f8acedd458bc71d24f: Deleting consensus metadata
I20260809 07:48:53.289295 25901 catalog_manager.cc:4841] TS f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023): tablet 8d78f49b409c4d7b9136830b9b2842f7 (table test-table [id=290b69bcbd16437db3aa57038dffe5ed]) successfully deleted
W20260809 07:48:53.289637 25901 catalog_manager.cc:4808] TS f00cd805fbc448f8acedd458bc71d24f (127.18.71.67:33023): delete failed for tablet 8d78f49b409c4d7b9136830b9b2842f7 because the tablet was not found. No further retry: Not found: Tablet not found: 8d78f49b409c4d7b9136830b9b2842f7
W20260809 07:48:53.292128 25903 connection.cc:535] client connection to 127.18.71.65:37833 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:48:53.292591 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 26316
I20260809 07:48:53.294924 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:53.292802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37833 (local address 127.0.0.1:38804)
0809 07:48:53.294740 (+ 1938us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:53.294822 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:37833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1817}
W20260809 07:48:53.295143 25903 catalog_manager.cc:4584] TS 29fdb83dc40c4a308ef7a8b27bdae12e (127.18.71.65:37833): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d78f49b409c4d7b9136830b9b2842f7: Network error: Client connection negotiation failed: client connection to 127.18.71.65:37833: connect: Connection refused (error 111)
I20260809 07:48:53.304127 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 26461
I20260809 07:48:53.311749 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 25773
I20260809 07:48:53.319589 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 25888
2026-08-09T07:48:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7355 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:53 dist-test-slave-10pc krb5kdc[26739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:53 dist-test-slave-10pc krb5kdc[26739](info): set up 2 sockets
Aug 09 07:48:53 dist-test-slave-10pc krb5kdc[26739](info): commencing operation
krb5kdc: starting...
W20260809 07:48:54.409888 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:54 dist-test-slave-10pc krb5kdc[26739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:54Z Disabled control of system clock
WARNING: no policy specified for oryx/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab.
Entry for principal oryx/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab.
I20260809 07:48:54.543838 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33409
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab
--principal=oryx/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:54.658493 26753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:54.658804 26753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:54.658881 26753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:54.663023 26753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:54.663118 26753 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 07:48:54.663151 26753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:54.663223 26753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:54.663282 26753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:54.663323 26753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:54.668664 26753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33409
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.126.keytab
--principal=oryx/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:54.669759 26753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:54.670846 26753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:54.679891 26758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:54.679984 26759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:54.680040 26753 server_base.cc:1034] running on GCE node
W20260809 07:48:54.679924 26761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:54.680904 26753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:54.682019 26753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:54.683182 26753 hybrid_clock.cc:648] HybridClock initialized: now 1786261734683162 us; error 32 us; skew 500 ppm
Aug 09 07:48:54 dist-test-slave-10pc krb5kdc[26739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261734, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:54.686002 26753 init.cc:377] Logged in from keytab as oryx/127.18.71.126@KRBTEST.COM (short username oryx)
I20260809 07:48:54.686878 26753 webserver.cc:460] Webserver started at http://127.18.71.126:38755/ using document root <none> and password file <none>
I20260809 07:48:54.687438 26753 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:54.687527 26753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:54.687778 26753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:54.689879 26753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "224e4ec19bf24f82b127e966bf650b77"
format_stamp: "Formatted at 2026-08-09 07:48:54 on dist-test-slave-10pc"
server_key: "8a332a88f18f5c2c60645909f159d476"
server_key_iv: "ee38a92b10aae02838fbe547f0bf961a"
server_key_version: "encryptionkey@0"
I20260809 07:48:54.690523 26753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "224e4ec19bf24f82b127e966bf650b77"
format_stamp: "Formatted at 2026-08-09 07:48:54 on dist-test-slave-10pc"
server_key: "8a332a88f18f5c2c60645909f159d476"
server_key_iv: "ee38a92b10aae02838fbe547f0bf961a"
server_key_version: "encryptionkey@0"
I20260809 07:48:54.694587 26753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 07:48:54.697244 26768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:54.698572 26753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:54.698731 26753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "224e4ec19bf24f82b127e966bf650b77"
format_stamp: "Formatted at 2026-08-09 07:48:54 on dist-test-slave-10pc"
server_key: "8a332a88f18f5c2c60645909f159d476"
server_key_iv: "ee38a92b10aae02838fbe547f0bf961a"
server_key_version: "encryptionkey@0"
I20260809 07:48:54.698849 26753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:54.719647 26753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:54.722954 26753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:54.723198 26753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:54.731401 26753 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33409
I20260809 07:48:54.731433 26839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33409 every 8 connection(s)
I20260809 07:48:54.732364 26753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:54.735214 26840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:54.740847 26840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Bootstrap starting.
I20260809 07:48:54.741259 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 26753
I20260809 07:48:54.741397 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:54.741704 18717 external_mini_cluster.cc:1428] Setting key a01900a2dba576064a4e7323db73fe5c
I20260809 07:48:54.743564 26840 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:54.744441 26840 log.cc:824] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:54.746794 26840 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: No bootstrap required, opened a new log
I20260809 07:48:54.750811 26840 raft_consensus.cc:348] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224e4ec19bf24f82b127e966bf650b77" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33409 } }
I20260809 07:48:54.751121 26840 raft_consensus.cc:374] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:54.751200 26840 raft_consensus.cc:729] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 224e4ec19bf24f82b127e966bf650b77, State: Initialized, Role: FOLLOWER
Aug 09 07:48:54 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:54.751731 26840 consensus_queue.cc:260] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224e4ec19bf24f82b127e966bf650b77" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33409 } }
I20260809 07:48:54.751899 26840 raft_consensus.cc:388] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:54.751976 26840 raft_consensus.cc:482] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:54.752094 26840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:54.753279 26840 raft_consensus.cc:504] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224e4ec19bf24f82b127e966bf650b77" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33409 } }
I20260809 07:48:54.753638 26840 leader_election.cc:302] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 224e4ec19bf24f82b127e966bf650b77; no voters:
I20260809 07:48:54.753974 26840 leader_election.cc:288] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:54.754053 26845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:54.754263 26845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [term 1 LEADER]: Becoming Leader. State: Replica: 224e4ec19bf24f82b127e966bf650b77, State: Running, Role: LEADER
I20260809 07:48:54.754583 26845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224e4ec19bf24f82b127e966bf650b77" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33409 } }
I20260809 07:48:54.755805 26840 sys_catalog.cc:564] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:54.756834 26846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 224e4ec19bf24f82b127e966bf650b77. Latest consensus state: current_term: 1 leader_uuid: "224e4ec19bf24f82b127e966bf650b77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224e4ec19bf24f82b127e966bf650b77" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33409 } } }
I20260809 07:48:54.756973 26846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:54.757237 26848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "224e4ec19bf24f82b127e966bf650b77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "224e4ec19bf24f82b127e966bf650b77" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33409 } } }
I20260809 07:48:54.757328 26848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:54.757591 26856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:54.757819 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:54.743090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38602 (local address 127.18.71.126:33409)
0809 07:48:54.743559 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:54.743581 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:54.743629 (+ 48us) server_negotiation.cc:407] Connection header received
0809 07:48:54.744516 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:54.744543 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:54.744980 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:54.745318 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:54.746043 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:54.747551 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:54.748408 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:54.749174 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:54.749345 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:54.752073 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:54.752079 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:54.752096 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:54.752144 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:54.754066 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:54.754578 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:54.754583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:54.754588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:54.754720 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:54.755080 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:54.755083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:54.755084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:54.755581 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:54.755794 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:54.756074 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 07:48:54.756401 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":121,"threads_started":1}
I20260809 07:48:54.760870 26856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:54.766445 26856 catalog_manager.cc:1347] Generated new cluster ID: 0ab8ed9b06f445d599da3ecc03f78aef
I20260809 07:48:54.766508 26856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:54.784291 26856 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:54.785286 26856 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:54.795574 26856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Generated new TSK 0
I20260809 07:48:54.796392 26856 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab.
Entry for principal oryx/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab.
I20260809 07:48:54.832435 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33409
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab
--principal=oryx/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:54.949116 26869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:54.949368 26869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:54.949410 26869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:54.953341 26869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:54.953397 26869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:54.953434 26869 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:54.953465 26869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:54.958794 26869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab
--principal=oryx/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:54.959739 26869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:54.960780 26869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:54.970402 26875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:54.970402 26874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:54.970539 26869 server_base.cc:1034] running on GCE node
W20260809 07:48:54.970777 26877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:54.971247 26869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:54.971918 26869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:54.973109 26869 hybrid_clock.cc:648] HybridClock initialized: now 1786261734973076 us; error 56 us; skew 500 ppm
Aug 09 07:48:54 dist-test-slave-10pc krb5kdc[26739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261734, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:54.976379 26869 init.cc:377] Logged in from keytab as oryx/127.18.71.65@KRBTEST.COM (short username oryx)
I20260809 07:48:54.977336 26869 webserver.cc:460] Webserver started at http://127.18.71.65:33751/ using document root <none> and password file <none>
I20260809 07:48:54.977876 26869 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:54.977936 26869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:54.978107 26869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:54.980183 26869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "ad481fad7b3241ba9252d10843dd736d"
format_stamp: "Formatted at 2026-08-09 07:48:54 on dist-test-slave-10pc"
server_key: "4c51ffec76e721c5aa274d9611351661"
server_key_iv: "5202fd304db3694e688a4214849aaedc"
server_key_version: "encryptionkey@0"
I20260809 07:48:54.980805 26869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "ad481fad7b3241ba9252d10843dd736d"
format_stamp: "Formatted at 2026-08-09 07:48:54 on dist-test-slave-10pc"
server_key: "4c51ffec76e721c5aa274d9611351661"
server_key_iv: "5202fd304db3694e688a4214849aaedc"
server_key_version: "encryptionkey@0"
I20260809 07:48:54.984568 26869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:48:54.986994 26884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:54.988245 26869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:54.988375 26869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "ad481fad7b3241ba9252d10843dd736d"
format_stamp: "Formatted at 2026-08-09 07:48:54 on dist-test-slave-10pc"
server_key: "4c51ffec76e721c5aa274d9611351661"
server_key_iv: "5202fd304db3694e688a4214849aaedc"
server_key_version: "encryptionkey@0"
I20260809 07:48:54.988552 26869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:55.004210 26869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:55.007725 26869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:55.007944 26869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:55.009589 26869 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:55.009667 26869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.009738 26869 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:55.009783 26869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261734, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:55.021085 26869 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:38813
I20260809 07:48:55.021235 27006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:38813 every 8 connection(s)
I20260809 07:48:55.022157 26869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:55.024195 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.010367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:51909 (local address 127.18.71.126:33409)
0809 07:48:55.010517 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.010529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.011501 (+ 972us) server_negotiation.cc:407] Connection header received
0809 07:48:55.012554 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.012558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.012617 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.012682 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.014186 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.014821 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.016632 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.017017 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.017133 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.020243 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:55.020246 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:55.020249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:55.020274 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:55.022006 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.022618 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.022622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.022624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.022723 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.023111 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.023115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.023117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.023360 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:55.023448 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.023910 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.024067 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:48:55.025012 26893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.010829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33409 (local address 127.18.71.65:51909)
0809 07:48:55.011322 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.011372 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.012266 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.013198 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.013204 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.013529 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:55.014031 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.014051 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.014960 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.014964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.016457 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.016492 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.017319 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.017322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.017461 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.018111 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:55.018133 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:55.020051 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:55.022131 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.022138 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.022152 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.022491 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.022836 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.022839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.022841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.023002 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.023459 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:55.023464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:55.023694 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.023849 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.024096 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":146,"threads_started":1}
I20260809 07:48:55.026306 27007 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33409
I20260809 07:48:55.026533 27007 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:55.027139 27007 heartbeater.cc:511] Master 127.18.71.126:33409 requested a full tablet report, sending...
I20260809 07:48:55.028927 26790 ts_manager.cc:194] Registered new tserver with Master: ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813)
I20260809 07:48:55.029951 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 26869
I20260809 07:48:55.030100 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:55.030156 26790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:51909
I20260809 07:48:55.030457 18717 external_mini_cluster.cc:1428] Setting key 667bd5c65ccd0bef800d67bc3b1f3c4b
I20260809 07:48:55.039718 27013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.028783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33409 (local address 127.18.71.65:59061)
0809 07:48:55.030941 (+ 2158us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.030957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.031059 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.032720 (+ 1661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.032723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.032927 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:55.033029 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.033035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.033795 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.033799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.034355 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.034362 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.034990 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.034993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.035077 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.035640 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:55.035652 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:55.035990 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:55.038137 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.038140 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.038141 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.038335 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.038635 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.038637 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.038638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.038712 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.039232 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:55.039235 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:55.039319 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.039428 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.039552 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2067,"thread_start_us":117,"threads_started":1}
I20260809 07:48:55.039901 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.028899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:59061 (local address 127.18.71.126:33409)
0809 07:48:55.029040 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.029045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.031019 (+ 1974us) server_negotiation.cc:407] Connection header received
0809 07:48:55.032555 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.032557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.032597 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.032690 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.033153 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.033656 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.034474 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.034864 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.034978 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.036150 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:55.036157 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:55.036163 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:55.036194 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:55.038010 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.038434 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.038440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.038443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.038538 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.038805 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.038811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.038814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.039084 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:55.039221 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.039520 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.039666 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
WARNING: no policy specified for oryx/127.18.71.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab.
Entry for principal oryx/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab.
I20260809 07:48:55.063630 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33409
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab
--principal=oryx/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:55.179561 27017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:55.179891 27017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:55.179973 27017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:55.184176 27017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:55.184239 27017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:55.184278 27017 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:55.184358 27017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:48:55.189692 27017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab
--principal=oryx/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:55.190760 27017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:55.191859 27017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:55.202280 27023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.202280 27024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.202376 27026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:55.202564 27017 server_base.cc:1034] running on GCE node
I20260809 07:48:55.203074 27017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:55.203810 27017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:55.205047 27017 hybrid_clock.cc:648] HybridClock initialized: now 1786261735205035 us; error 30 us; skew 500 ppm
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261735, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:55.208029 27017 init.cc:377] Logged in from keytab as oryx/127.18.71.66@KRBTEST.COM (short username oryx)
I20260809 07:48:55.209019 27017 webserver.cc:460] Webserver started at http://127.18.71.66:39159/ using document root <none> and password file <none>
I20260809 07:48:55.209589 27017 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:55.209648 27017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:55.209826 27017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:55.211943 27017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "9981c18e922e4c62bd0f3fd6a22aa904"
format_stamp: "Formatted at 2026-08-09 07:48:55 on dist-test-slave-10pc"
server_key: "4be2aaec13c329f781250d4b97009b01"
server_key_iv: "a6c2406f2990802f66fc134a964ad106"
server_key_version: "encryptionkey@0"
I20260809 07:48:55.212568 27017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "9981c18e922e4c62bd0f3fd6a22aa904"
format_stamp: "Formatted at 2026-08-09 07:48:55 on dist-test-slave-10pc"
server_key: "4be2aaec13c329f781250d4b97009b01"
server_key_iv: "a6c2406f2990802f66fc134a964ad106"
server_key_version: "encryptionkey@0"
I20260809 07:48:55.216531 27017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:48:55.218999 27033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.220232 27017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:55.220340 27017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "9981c18e922e4c62bd0f3fd6a22aa904"
format_stamp: "Formatted at 2026-08-09 07:48:55 on dist-test-slave-10pc"
server_key: "4be2aaec13c329f781250d4b97009b01"
server_key_iv: "a6c2406f2990802f66fc134a964ad106"
server_key_version: "encryptionkey@0"
I20260809 07:48:55.220427 27017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:55.250109 27017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:55.253422 27017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:55.253602 27017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:55.255044 27017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:55.255129 27017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.255203 27017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:55.255255 27017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261735, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:55.267148 27017 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:46599
I20260809 07:48:55.267207 27157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:46599 every 8 connection(s)
I20260809 07:48:55.268359 27017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:48:55.271070 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.255977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:34497 (local address 127.18.71.126:33409)
0809 07:48:55.256144 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.256151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.257223 (+ 1072us) server_negotiation.cc:407] Connection header received
0809 07:48:55.258617 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.258623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.258690 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.258798 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.260561 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.261545 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.262858 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.263155 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.263271 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.266339 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:55.266343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:55.266346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:55.266384 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:55.268156 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.268908 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.268915 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.268920 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.269024 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.269503 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.269507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.269511 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.269775 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:55.269884 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.270667 (+ 783us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.270834 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 07:48:55.271678 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 27017
I20260809 07:48:55.271823 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:48:55.272176 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.256560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33409 (local address 127.18.71.66:34497)
0809 07:48:55.257035 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.257086 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.258347 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.259040 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.259049 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.259514 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:55.260302 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.260325 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.261695 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.261698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.262713 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.262723 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.263331 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.263336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.263518 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.264179 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:55.264206 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:55.266172 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:55.268317 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.268323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.268339 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.268774 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.269158 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.269163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.269168 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.269375 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.269915 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:55.269923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:55.270308 (+ 385us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.270553 (+ 245us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.270988 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":123,"threads_started":1}
I20260809 07:48:55.272205 18717 external_mini_cluster.cc:1428] Setting key 61c880c639e903ddab0f2761bd2ab12b
I20260809 07:48:55.276880 27158 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33409
I20260809 07:48:55.277045 27158 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:55.277513 27158 heartbeater.cc:511] Master 127.18.71.126:33409 requested a full tablet report, sending...
I20260809 07:48:55.278827 26790 ts_manager.cc:194] Registered new tserver with Master: 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:55.279757 26790 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:34497
WARNING: no policy specified for oryx/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.67@KRBTEST.COM" created.
I20260809 07:48:55.291527 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.278465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:59793 (local address 127.18.71.126:33409)
0809 07:48:55.278594 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.278601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.282569 (+ 3968us) server_negotiation.cc:407] Connection header received
0809 07:48:55.282763 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.282766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.282814 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.282862 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.283584 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.284444 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.285707 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.285942 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.286097 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.287352 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:55.287369 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:55.287371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:55.287392 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:55.289177 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.289747 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.289752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.289754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.289850 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.290280 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.290287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.290292 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.290641 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:55.290751 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.291122 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.291267 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 07:48:55.292040 27166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.278341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33409 (local address 127.18.71.66:59793)
0809 07:48:55.282477 (+ 4136us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.282493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.282649 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.283021 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.283025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.283286 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:55.283437 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.283445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.284654 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.284657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.285572 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.285582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.286107 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.286110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.286215 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.286774 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:55.286795 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:55.287183 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:55.289316 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.289318 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.289320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.289625 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.290027 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.290030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.290033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.290145 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.290804 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:55.290809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:55.290923 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.291759 (+ 836us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.291898 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4034,"thread_start_us":96,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab.
Entry for principal oryx/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab.
I20260809 07:48:55.307588 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33409
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab
--principal=oryx/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:55.433825 27168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:55.434089 27168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:55.434134 27168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:55.438169 27168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:55.438227 27168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:55.438266 27168 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 07:48:55.438345 27168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:48:55.443591 27168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab
--principal=oryx/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.18.71.126:33409
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:55.444710 27168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:55.445745 27168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:55.455720 27177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.455721 27174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:55.455806 27175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:55.456213 27168 server_base.cc:1034] running on GCE node
I20260809 07:48:55.456832 27168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:55.457505 27168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:55.458671 27168 hybrid_clock.cc:648] HybridClock initialized: now 1786261735458657 us; error 24 us; skew 500 ppm
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261735, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:55.461500 27168 init.cc:377] Logged in from keytab as oryx/127.18.71.67@KRBTEST.COM (short username oryx)
I20260809 07:48:55.462389 27168 webserver.cc:460] Webserver started at http://127.18.71.67:33595/ using document root <none> and password file <none>
I20260809 07:48:55.462961 27168 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:55.463025 27168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:55.463250 27168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:55.465389 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "b15a6406d23c4e1090d411f41bd664c2"
format_stamp: "Formatted at 2026-08-09 07:48:55 on dist-test-slave-10pc"
server_key: "c378577e93ee114cfe2a8111b974afe4"
server_key_iv: "f2bdbf94fa78843c252998345c663d15"
server_key_version: "encryptionkey@0"
I20260809 07:48:55.465979 27168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "b15a6406d23c4e1090d411f41bd664c2"
format_stamp: "Formatted at 2026-08-09 07:48:55 on dist-test-slave-10pc"
server_key: "c378577e93ee114cfe2a8111b974afe4"
server_key_iv: "f2bdbf94fa78843c252998345c663d15"
server_key_version: "encryptionkey@0"
I20260809 07:48:55.469686 27168 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:48:55.472157 27184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.473393 27168 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:55.473539 27168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "b15a6406d23c4e1090d411f41bd664c2"
format_stamp: "Formatted at 2026-08-09 07:48:55 on dist-test-slave-10pc"
server_key: "c378577e93ee114cfe2a8111b974afe4"
server_key_iv: "f2bdbf94fa78843c252998345c663d15"
server_key_version: "encryptionkey@0"
I20260809 07:48:55.473680 27168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:55.496605 27168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:55.499877 27168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:55.500098 27168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:55.501631 27168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:55.501708 27168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:55.501842 27168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:55.501889 27168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261735, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:55.513211 27168 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:44651
I20260809 07:48:55.513276 27310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:44651 every 8 connection(s)
I20260809 07:48:55.514122 27168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:48:55.516654 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 27168
I20260809 07:48:55.516809 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:48:55.517131 18717 external_mini_cluster.cc:1428] Setting key e9527d54b9c43b66d400ab3b935e85ce
I20260809 07:48:55.517892 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.502640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:44435 (local address 127.18.71.126:33409)
0809 07:48:55.502802 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.502807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.503862 (+ 1055us) server_negotiation.cc:407] Connection header received
0809 07:48:55.505240 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.505247 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.505308 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.505359 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.507415 (+ 2056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.508419 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.509924 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.510197 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.510328 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.513621 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:55.513632 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:55.513636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:55.513664 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:55.515392 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.516037 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.516041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.516043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.516142 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.516569 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.516572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.516574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.516812 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:55.516944 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.517560 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.517721 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 07:48:55.518919 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.503156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33409 (local address 127.18.71.67:44435)
0809 07:48:55.503706 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.503761 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.504925 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.505895 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.505904 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.506396 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:55.507238 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.507266 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.508578 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.508582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.509777 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.509789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.510373 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.510377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.510586 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.511255 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:55.511279 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:55.513418 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:55.515532 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.515539 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.515556 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.515911 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.516245 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.516250 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.516252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.516431 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.516956 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:55.516976 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:55.517251 (+ 275us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.517471 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.517839 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":130,"threads_started":1}
I20260809 07:48:55.520177 27311 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33409
I20260809 07:48:55.520325 27311 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:55.520953 27311 heartbeater.cc:511] Master 127.18.71.126:33409 requested a full tablet report, sending...
I20260809 07:48:55.522138 26789 ts_manager.cc:194] Registered new tserver with Master: b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67:44651)
I20260809 07:48:55.522972 26789 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:44435
I20260809 07:48:55.530115 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.522741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33409 (local address 127.18.71.67:59187)
0809 07:48:55.523066 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.523106 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.523207 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.523490 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.523493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.523688 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:55.523826 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.523834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.524593 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.524597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.525563 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.525573 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.526052 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.526055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.526131 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.526650 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:55.526662 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:55.527052 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:55.528776 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.528779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.528780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.528970 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.529209 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:55.529211 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:55.529212 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:55.529285 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:55.529714 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:55.529716 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:55.529787 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.529864 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.529967 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":100,"threads_started":1}
I20260809 07:48:55.530442 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.522858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:59187 (local address 127.18.71.126:33409)
0809 07:48:55.522987 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.522994 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.523190 (+ 196us) server_negotiation.cc:407] Connection header received
0809 07:48:55.523339 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.523342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.523391 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.523473 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.523928 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.524433 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.525675 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.525882 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.525962 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.527177 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:55.527180 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:55.527182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:55.527228 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:55.528668 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.529055 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.529057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.529058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.529132 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.529393 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.529396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.529397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.529615 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:55.529731 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.530141 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.530262 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:48:55.531376 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261735, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 07:48:55.557060 26843 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:55.550090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.18.71.126:33409)
0809 07:48:55.550246 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.550250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.550263 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:48:55.550307 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.550310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.550350 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.550450 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.551284 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.551864 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.552886 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.553130 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.553255 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.556871 (+ 3616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38612: BlockingRecv error: recv got EOF from 127.0.0.1:38612 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 07:48:55 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261735, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:55.566898 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.558712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.18.71.126:33409)
0809 07:48:55.558849 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.558854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.559189 (+ 335us) server_negotiation.cc:407] Connection header received
0809 07:48:55.559266 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.559269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.559310 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.559380 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:55.559945 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.560555 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.561333 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.561590 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.561671 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.563881 (+ 2210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:55.563884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:55.563886 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:55.563910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:55.565451 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.565832 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.565834 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.565835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.565911 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:55.566134 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:55.566136 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:55.566138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:55.566303 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:55.566403 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.566567 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.566662 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:48:55.569919 26789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:55.573032 26789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:55.588426 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.582649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40926 (local address 127.18.71.66:46599)
0809 07:48:55.583481 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.583494 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.583531 (+ 37us) server_negotiation.cc:407] Connection header received
0809 07:48:55.583615 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.583623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.583848 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.584381 (+ 533us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:55.584591 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.585295 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.587189 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.587728 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.587851 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.588038 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.588162 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.588259 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":114,"threads_started":1}
I20260809 07:48:55.588416 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.581839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:46599 (local address 127.0.0.1:40926)
0809 07:48:55.582240 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.582299 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.582426 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.584029 (+ 1603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.584032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.584052 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:55.584243 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.584248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.585418 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.585424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.587041 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.587048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.587863 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.587867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.587974 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.588015 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.588131 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.588260 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":194,"spinlock_wait_cycles":384,"thread_start_us":111,"threads_started":1}
I20260809 07:48:55.588625 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.581839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:38813 (local address 127.0.0.1:49376)
0809 07:48:55.582673 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.582688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.582782 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.583720 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.583727 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.583751 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:55.583914 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.583923 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.585544 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.585547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.587028 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.587042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.588035 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.588038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.588134 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.588152 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.588196 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.588508 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":384,"spinlock_wait_cycles":2432,"thread_start_us":102,"threads_started":1}
I20260809 07:48:55.588718 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.582130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.18.71.65:38813)
0809 07:48:55.583206 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.583214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.583240 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:48:55.583323 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.583328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.583549 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.584062 (+ 513us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:55.584214 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.585425 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.587174 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.587911 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.588277 (+ 366us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.588418 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.588494 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.588580 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":934,"thread_start_us":105,"threads_started":1}
I20260809 07:48:55.588421 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.582647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:44651 (local address 127.0.0.1:45788)
0809 07:48:55.582885 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.582898 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.582972 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.583978 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.583983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.584016 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:55.584241 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.584248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.585678 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.585684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.586798 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.586808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.587907 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.587911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.588028 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.588055 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.588102 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.588269 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":44,"spinlock_wait_cycles":4864,"thread_start_us":104,"threads_started":1}
I20260809 07:48:55.589352 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.582989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45788 (local address 127.18.71.67:44651)
0809 07:48:55.583493 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.583505 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.583528 (+ 23us) server_negotiation.cc:407] Connection header received
0809 07:48:55.583600 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.583607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.583815 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.583998 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:55.584367 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.585555 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.586942 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.587741 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.588912 (+ 1171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.589092 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.589140 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.589223 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":128,"threads_started":1}
I20260809 07:48:55.591868 26938 tablet_service.cc:1467] Processing CreateTablet for tablet 4fefb6ac0c194c3083f837459d4a57b1 (DEFAULT_TABLE table=test-table [id=98f135563ca840e6b10dac57a702f225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:55.591938 27238 tablet_service.cc:1467] Processing CreateTablet for tablet 4fefb6ac0c194c3083f837459d4a57b1 (DEFAULT_TABLE table=test-table [id=98f135563ca840e6b10dac57a702f225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:55.591938 27087 tablet_service.cc:1467] Processing CreateTablet for tablet 4fefb6ac0c194c3083f837459d4a57b1 (DEFAULT_TABLE table=test-table [id=98f135563ca840e6b10dac57a702f225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:55.592926 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fefb6ac0c194c3083f837459d4a57b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:55.592954 27238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fefb6ac0c194c3083f837459d4a57b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:55.592954 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fefb6ac0c194c3083f837459d4a57b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:55.599972 27341 tablet_bootstrap.cc:492] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Bootstrap starting.
I20260809 07:48:55.601538 27344 tablet_bootstrap.cc:492] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Bootstrap starting.
I20260809 07:48:55.601945 27342 tablet_bootstrap.cc:492] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Bootstrap starting.
I20260809 07:48:55.602422 27341 tablet_bootstrap.cc:654] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:55.603251 27341 log.cc:824] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:55.603770 27344 tablet_bootstrap.cc:654] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:55.603941 27342 tablet_bootstrap.cc:654] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:55.604663 27344 log.cc:824] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:55.604712 27342 log.cc:824] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:55.605808 27341 tablet_bootstrap.cc:492] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: No bootstrap required, opened a new log
I20260809 07:48:55.606024 27341 ts_tablet_manager.cc:1397] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:48:55.606513 27342 tablet_bootstrap.cc:492] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: No bootstrap required, opened a new log
I20260809 07:48:55.606706 27342 ts_tablet_manager.cc:1397] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:48:55.609848 27344 tablet_bootstrap.cc:492] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: No bootstrap required, opened a new log
I20260809 07:48:55.609716 27341 raft_consensus.cc:348] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.610005 27341 raft_consensus.cc:374] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:55.610070 27344 ts_tablet_manager.cc:1397] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260809 07:48:55.610071 27341 raft_consensus.cc:729] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad481fad7b3241ba9252d10843dd736d, State: Initialized, Role: FOLLOWER
I20260809 07:48:55.609977 27342 raft_consensus.cc:348] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.610239 27342 raft_consensus.cc:374] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:55.610312 27342 raft_consensus.cc:729] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Initialized, Role: FOLLOWER
I20260809 07:48:55.610639 27341 consensus_queue.cc:260] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.610749 27342 consensus_queue.cc:260] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.611706 27341 ts_tablet_manager.cc:1428] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:48:55.611717 27342 ts_tablet_manager.cc:1428] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 07:48:55.612080 27007 heartbeater.cc:503] Master 127.18.71.126:33409 was elected leader, sending a full tablet report...
I20260809 07:48:55.612185 27311 heartbeater.cc:503] Master 127.18.71.126:33409 was elected leader, sending a full tablet report...
I20260809 07:48:55.614089 27344 raft_consensus.cc:348] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.614362 27344 raft_consensus.cc:374] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:55.614432 27344 raft_consensus.cc:729] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9981c18e922e4c62bd0f3fd6a22aa904, State: Initialized, Role: FOLLOWER
I20260809 07:48:55.614996 27344 consensus_queue.cc:260] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.615921 27349 raft_consensus.cc:482] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:55.616050 27344 ts_tablet_manager.cc:1428] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:48:55.616080 27349 raft_consensus.cc:504] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.616609 27158 heartbeater.cc:503] Master 127.18.71.126:33409 was elected leader, sending a full tablet report...
I20260809 07:48:55.617255 27349 leader_election.cc:288] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:55.621783 27194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.617428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:38813 (local address 127.18.71.67:46879)
0809 07:48:55.617677 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.617694 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.617776 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.618165 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.618171 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.618190 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:55.618319 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.618327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.619218 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.619222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.620250 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.620259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.621341 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.621344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.621476 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.621501 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.621583 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.621656 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 07:48:55.621798 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.617747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:46879 (local address 127.18.71.65:38813)
0809 07:48:55.617881 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.617886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.617901 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:55.617945 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.617948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.618012 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.618103 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:55.618446 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.619102 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.620556 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.621212 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.621328 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.621398 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.621633 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.621711 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 07:48:55.622422 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fefb6ac0c194c3083f837459d4a57b1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d" is_pre_election: true
I20260809 07:48:55.622593 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.617510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:46599 (local address 127.18.71.67:41101)
0809 07:48:55.618020 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:55.618031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:55.618147 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:55.618508 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:55.618511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:55.618527 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:55.618629 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.618636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.619425 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.619427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.621338 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:55.621373 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.622214 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.622217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:55.622316 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.622339 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:48:55.622412 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 07:48:55.622462 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":127,"threads_started":1}
I20260809 07:48:55.622649 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.617639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:41101 (local address 127.18.71.66:46599)
0809 07:48:55.617868 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.617872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.618111 (+ 239us) server_negotiation.cc:407] Connection header received
0809 07:48:55.618261 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.618264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.618308 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.618357 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:55.618745 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.619313 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.621476 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.622081 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.622216 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.622260 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.622461 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.622542 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260809 07:48:55.622727 26958 raft_consensus.cc:2454] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:55.623204 27185 leader_election.cc:302] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad481fad7b3241ba9252d10843dd736d, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:55.623492 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fefb6ac0c194c3083f837459d4a57b1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" is_pre_election: true
I20260809 07:48:55.623538 27349 raft_consensus.cc:2781] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:55.623626 27349 raft_consensus.cc:482] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:55.623684 27349 raft_consensus.cc:3037] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:55.623762 27109 raft_consensus.cc:2454] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:55.625043 27349 raft_consensus.cc:504] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.625385 27349 leader_election.cc:288] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Requested vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:55.625699 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fefb6ac0c194c3083f837459d4a57b1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d"
I20260809 07:48:55.625746 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fefb6ac0c194c3083f837459d4a57b1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904"
I20260809 07:48:55.625821 26958 raft_consensus.cc:3037] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:55.625888 27109 raft_consensus.cc:3037] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:55.626950 26958 raft_consensus.cc:2454] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:55.627256 27185 leader_election.cc:302] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad481fad7b3241ba9252d10843dd736d, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:55.627399 27109 raft_consensus.cc:2454] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:55.627434 27349 raft_consensus.cc:2781] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:55.627661 27349 raft_consensus.cc:686] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Becoming Leader. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Running, Role: LEADER
I20260809 07:48:55.627983 27349 consensus_queue.cc:237] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.631098 26789 catalog_manager.cc:5495] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "b15a6406d23c4e1090d411f41bd664c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } health_report { overall_health: HEALTHY } } }
I20260809 07:48:55.651854 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:55.648146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.18.71.67:44651)
0809 07:48:55.648290 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:55.648295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:55.648311 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:48:55.648484 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:55.648487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:55.648547 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:55.648672 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:55.648945 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.649456 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.650335 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:55.650560 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:55.650679 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:55.650972 (+ 293us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:55.651433 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:55.651565 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:55.651640 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 07:48:55.651700 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:48:55.658110 27109 raft_consensus.cc:1261] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:55.658135 26958 raft_consensus.cc:1261] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:55.658948 27349 consensus_queue.cc:1035] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:55.659204 27352 consensus_queue.cc:1035] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:55.669148 27355 mvcc.cc:205] Tried to move back new op lower bound from 7316528069247078400 to 7316528069135204352. Current Snapshot: MvccSnapshot[applied={T|T < 7316528069247078400 or (T in {7316528069247078400})}]
I20260809 07:48:55.671455 27357 mvcc.cc:205] Tried to move back new op lower bound from 7316528069247078400 to 7316528069135204352. Current Snapshot: MvccSnapshot[applied={T|T < 7316528069247078400 or (T in {7316528069247078400})}]
I20260809 07:48:55.675302 26789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:55.675462 26789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:55.678153 26789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Sending DeleteTablet for 3 replicas of tablet 4fefb6ac0c194c3083f837459d4a57b1
I20260809 07:48:55.678959 26938 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fefb6ac0c194c3083f837459d4a57b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:55 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:49376
I20260809 07:48:55.679507 27366 tablet_replica.cc:330] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: stopping tablet replica
I20260809 07:48:55.679548 26789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 07:48:55.679828 27366 raft_consensus.cc:2229] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 07:48:55.679947 26789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:55.680001 27087 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fefb6ac0c194c3083f837459d4a57b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:55 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40926
I20260809 07:48:55.680157 27366 raft_consensus.cc:2258] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:55.680498 27238 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fefb6ac0c194c3083f837459d4a57b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:55 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:45788
I20260809 07:48:55.680933 27367 tablet_replica.cc:330] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: stopping tablet replica
I20260809 07:48:55.681241 27367 raft_consensus.cc:2229] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:55.681530 27367 raft_consensus.cc:2258] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:55.681866 27368 tablet_replica.cc:330] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: stopping tablet replica
I20260809 07:48:55.682132 27368 raft_consensus.cc:2229] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:55.682466 27366 ts_tablet_manager.cc:1905] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:55.683363 27368 raft_consensus.cc:2258] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:55.684862 27367 ts_tablet_manager.cc:1905] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:55.685618 27087 tablet_service.cc:1467] Processing CreateTablet for tablet 85f3de6584af4ab6955438ac6fcd146a (DEFAULT_TABLE table=test-table [id=f318863b00ed4ed9afec46ecf1b66946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:55.685905 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85f3de6584af4ab6955438ac6fcd146a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:55.687302 27366 ts_tablet_manager.cc:1918] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:55.687484 27366 log.cc:1196] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/4fefb6ac0c194c3083f837459d4a57b1
I20260809 07:48:55.687621 27367 ts_tablet_manager.cc:1918] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:55.687726 27367 log.cc:1196] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/4fefb6ac0c194c3083f837459d4a57b1
I20260809 07:48:55.688023 27366 ts_tablet_manager.cc:1939] T 4fefb6ac0c194c3083f837459d4a57b1 P ad481fad7b3241ba9252d10843dd736d: Deleting consensus metadata
I20260809 07:48:55.688056 27367 ts_tablet_manager.cc:1939] T 4fefb6ac0c194c3083f837459d4a57b1 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting consensus metadata
I20260809 07:48:55.688421 27238 tablet_service.cc:1467] Processing CreateTablet for tablet 85f3de6584af4ab6955438ac6fcd146a (DEFAULT_TABLE table=test-table [id=f318863b00ed4ed9afec46ecf1b66946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:55.688732 27238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85f3de6584af4ab6955438ac6fcd146a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:55.689078 27368 ts_tablet_manager.cc:1905] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:55.690899 27342 tablet_bootstrap.cc:492] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Bootstrap starting.
I20260809 07:48:55.691335 26770 catalog_manager.cc:4841] TS ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813): tablet 4fefb6ac0c194c3083f837459d4a57b1 (table test-table [id=98f135563ca840e6b10dac57a702f225]) successfully deleted
I20260809 07:48:55.691545 27368 ts_tablet_manager.cc:1918] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:55.691628 27368 log.cc:1196] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/4fefb6ac0c194c3083f837459d4a57b1
I20260809 07:48:55.691951 27342 tablet_bootstrap.cc:654] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:55.692072 26938 tablet_service.cc:1467] Processing CreateTablet for tablet 85f3de6584af4ab6955438ac6fcd146a (DEFAULT_TABLE table=test-table [id=f318863b00ed4ed9afec46ecf1b66946]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:55.692320 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85f3de6584af4ab6955438ac6fcd146a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:55.693545 27342 tablet_bootstrap.cc:492] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: No bootstrap required, opened a new log
I20260809 07:48:55.693657 27342 ts_tablet_manager.cc:1397] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:55.694065 27342 raft_consensus.cc:348] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.694187 27342 raft_consensus.cc:374] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:55.694228 27342 raft_consensus.cc:729] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Initialized, Role: FOLLOWER
I20260809 07:48:55.694355 27342 consensus_queue.cc:260] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.694564 27341 tablet_bootstrap.cc:492] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Bootstrap starting.
I20260809 07:48:55.694664 27342 ts_tablet_manager.cc:1428] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:55.694914 27368 ts_tablet_manager.cc:1939] T 4fefb6ac0c194c3083f837459d4a57b1 P b15a6406d23c4e1090d411f41bd664c2: Deleting consensus metadata
I20260809 07:48:55.695597 27341 tablet_bootstrap.cc:654] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:55.696075 26773 catalog_manager.cc:4841] TS b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67:44651): tablet 4fefb6ac0c194c3083f837459d4a57b1 (table test-table [id=98f135563ca840e6b10dac57a702f225]) successfully deleted
I20260809 07:48:55.696560 27344 tablet_bootstrap.cc:492] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Bootstrap starting.
I20260809 07:48:55.697549 27344 tablet_bootstrap.cc:654] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:55.698311 27341 tablet_bootstrap.cc:492] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: No bootstrap required, opened a new log
I20260809 07:48:55.698408 27341 ts_tablet_manager.cc:1397] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 07:48:55.699249 27341 raft_consensus.cc:348] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.699371 27341 raft_consensus.cc:374] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:55.699391 27341 raft_consensus.cc:729] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad481fad7b3241ba9252d10843dd736d, State: Initialized, Role: FOLLOWER
I20260809 07:48:55.699417 27344 tablet_bootstrap.cc:492] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: No bootstrap required, opened a new log
I20260809 07:48:55.699496 27344 ts_tablet_manager.cc:1397] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:55.699530 27341 consensus_queue.cc:260] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.699915 27344 raft_consensus.cc:348] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.700034 27344 raft_consensus.cc:374] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:55.700042 27341 ts_tablet_manager.cc:1428] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:55.700066 27344 raft_consensus.cc:729] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9981c18e922e4c62bd0f3fd6a22aa904, State: Initialized, Role: FOLLOWER
I20260809 07:48:55.700189 27344 consensus_queue.cc:260] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.700433 27344 ts_tablet_manager.cc:1428] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:55.703696 26773 catalog_manager.cc:4841] TS 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599): tablet 4fefb6ac0c194c3083f837459d4a57b1 (table test-table [id=98f135563ca840e6b10dac57a702f225]) successfully deleted
W20260809 07:48:55.765973 27312 tablet.cc:2369] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:48:55.770176 27159 tablet.cc:2369] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:48:55.773725 27008 tablet.cc:2369] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:48:55.950047 27362 raft_consensus.cc:482] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:55.950207 27362 raft_consensus.cc:504] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.950750 27362 leader_election.cc:288] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:55.951269 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85f3de6584af4ab6955438ac6fcd146a" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" is_pre_election: true
I20260809 07:48:55.951274 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85f3de6584af4ab6955438ac6fcd146a" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d" is_pre_election: true
I20260809 07:48:55.951412 27109 raft_consensus.cc:2454] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:55.951448 26958 raft_consensus.cc:2454] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:55.951756 27188 leader_election.cc:302] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9981c18e922e4c62bd0f3fd6a22aa904, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:55.951947 27362 raft_consensus.cc:2781] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:55.952008 27362 raft_consensus.cc:482] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:55.952036 27362 raft_consensus.cc:3037] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:55.953097 27362 raft_consensus.cc:504] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.953409 27362 leader_election.cc:288] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Requested vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:55.953691 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85f3de6584af4ab6955438ac6fcd146a" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d"
I20260809 07:48:55.953759 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85f3de6584af4ab6955438ac6fcd146a" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904"
I20260809 07:48:55.953833 26958 raft_consensus.cc:3037] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:55.953886 27109 raft_consensus.cc:3037] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:55.954805 26958 raft_consensus.cc:2454] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:55.954839 27109 raft_consensus.cc:2454] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:55.955101 27185 leader_election.cc:302] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad481fad7b3241ba9252d10843dd736d, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:55.955308 27362 raft_consensus.cc:2781] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:55.955395 27362 raft_consensus.cc:686] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Becoming Leader. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Running, Role: LEADER
I20260809 07:48:55.955564 27362 consensus_queue.cc:237] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:55.957516 26789 catalog_manager.cc:5495] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "b15a6406d23c4e1090d411f41bd664c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } health_report { overall_health: HEALTHY } } }
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261734, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.126@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:56.015403 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.006252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33409 (local address 127.0.0.1:38624)
0809 07:48:56.006558 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:56.006571 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:56.006713 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:56.006962 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:56.006964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:56.007413 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:56.007521 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.007528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.008288 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.008293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.008910 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.008919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.009365 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.009369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.009467 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.010203 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:56.010218 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:56.011888 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:56.013834 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:56.013843 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:56.013848 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:56.014140 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:56.014397 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:56.014399 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:56.014400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:56.014484 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:56.014838 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:56.014843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:56.014952 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 07:48:56.015075 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 07:48:56.015210 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":109,"threads_started":1}
I20260809 07:48:56.015406 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.006306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.18.71.126:33409)
0809 07:48:56.006464 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.006471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.006648 (+ 177us) server_negotiation.cc:407] Connection header received
0809 07:48:56.006810 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.006813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.006859 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.006943 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:56.007656 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.008179 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.009027 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.009249 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.009360 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.012043 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:56.012046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:56.012048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:56.012072 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:56.013715 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:56.014233 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:56.014236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:56.014237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:56.014314 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:56.014565 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:56.014567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:56.014568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:56.014746 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:56.014822 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.015141 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.015276 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":52}
I20260809 07:48:56.019723 26789 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:38624:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 07:48:56.020613 26789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:56.025918 26938 tablet_service.cc:1467] Processing CreateTablet for tablet d63773ed0add4b6f90985b12c195afbe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ee4738a5d5e47048041ee5dbdbe37ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:56.025992 27087 tablet_service.cc:1467] Processing CreateTablet for tablet d63773ed0add4b6f90985b12c195afbe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ee4738a5d5e47048041ee5dbdbe37ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:56.026340 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63773ed0add4b6f90985b12c195afbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.026340 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63773ed0add4b6f90985b12c195afbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.029469 27344 tablet_bootstrap.cc:492] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: Bootstrap starting.
I20260809 07:48:56.029879 27238 tablet_service.cc:1467] Processing CreateTablet for tablet d63773ed0add4b6f90985b12c195afbe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8ee4738a5d5e47048041ee5dbdbe37ac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 07:48:56.030260 27238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d63773ed0add4b6f90985b12c195afbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.030428 27344 tablet_bootstrap.cc:654] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.031068 27341 tablet_bootstrap.cc:492] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: Bootstrap starting.
I20260809 07:48:56.031826 27341 tablet_bootstrap.cc:654] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.032809 27344 tablet_bootstrap.cc:492] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: No bootstrap required, opened a new log
I20260809 07:48:56.032894 27344 ts_tablet_manager.cc:1397] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 07:48:56.033092 27342 tablet_bootstrap.cc:492] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: Bootstrap starting.
I20260809 07:48:56.033313 27344 raft_consensus.cc:348] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.033414 27344 raft_consensus.cc:374] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.033440 27344 raft_consensus.cc:729] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9981c18e922e4c62bd0f3fd6a22aa904, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.033550 27344 consensus_queue.cc:260] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.033774 27344 ts_tablet_manager.cc:1428] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:56.034031 27342 tablet_bootstrap.cc:654] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.034248 27341 tablet_bootstrap.cc:492] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: No bootstrap required, opened a new log
I20260809 07:48:56.034317 27341 ts_tablet_manager.cc:1397] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 07:48:56.033777 27350 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.034468 27350 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:56.034693 27341 raft_consensus.cc:348] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.034801 27341 raft_consensus.cc:374] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.034832 27341 raft_consensus.cc:729] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad481fad7b3241ba9252d10843dd736d, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.034950 27341 consensus_queue.cc:260] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.035195 27348 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.035353 27348 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:56.035218 27341 ts_tablet_manager.cc:1428] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:56.037501 27342 tablet_bootstrap.cc:492] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: No bootstrap required, opened a new log
I20260809 07:48:56.037603 27342 ts_tablet_manager.cc:1397] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 07:48:56.038009 27342 raft_consensus.cc:348] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.038137 27342 raft_consensus.cc:374] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.038172 27342 raft_consensus.cc:729] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.038322 27342 consensus_queue.cc:260] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.038550 27342 ts_tablet_manager.cc:1428] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:56.038589 27362 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.038748 27362 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:56.160140 27350 raft_consensus.cc:482] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:56.160286 27350 raft_consensus.cc:504] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.161444 27350 leader_election.cc:288] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67:44651)
I20260809 07:48:56.165771 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.161830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:58433 (local address 127.18.71.67:44651)
0809 07:48:56.162258 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.162261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.162542 (+ 281us) server_negotiation.cc:407] Connection header received
0809 07:48:56.162650 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.162653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.162703 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.162811 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:56.163091 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.163623 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.164745 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.165171 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.165307 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.165359 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.165565 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.165640 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":71,"threads_started":1}
I20260809 07:48:56.165772 27385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.161830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:37535 (local address 127.18.71.65:38813)
0809 07:48:56.162125 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.162129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.162194 (+ 65us) server_negotiation.cc:407] Connection header received
0809 07:48:56.162368 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.162373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.162427 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.162525 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:48:56.162904 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.163502 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.164706 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.165154 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.165277 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.165335 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.165537 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.165625 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":74,"threads_started":1}
I20260809 07:48:56.165793 27383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.161725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:38813 (local address 127.18.71.66:37535)
0809 07:48:56.162090 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:56.162104 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:56.162246 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:56.162598 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:56.162604 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:56.162631 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:56.162779 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.162790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.163625 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.163629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.164582 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.164593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.165282 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.165286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.165397 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.165428 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 07:48:56.165491 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 07:48:56.165611 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":4,"thread_start_us":112,"threads_started":1}
I20260809 07:48:56.165800 27384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.161725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:44651 (local address 127.18.71.66:58433)
0809 07:48:56.162454 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:56.162469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:56.162552 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:56.162828 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:56.162830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:56.162866 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:48:56.162981 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.162987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.163735 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.163738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.164590 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.164599 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.165319 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.165322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.165422 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.165439 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 07:48:56.165532 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 07:48:56.165646 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":68,"thread_start_us":70,"threads_started":1}
I20260809 07:48:56.166373 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63773ed0add4b6f90985b12c195afbe" candidate_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d" is_pre_election: true
I20260809 07:48:56.166553 26958 raft_consensus.cc:2454] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9981c18e922e4c62bd0f3fd6a22aa904 in term 0.
I20260809 07:48:56.166798 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63773ed0add4b6f90985b12c195afbe" candidate_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b15a6406d23c4e1090d411f41bd664c2" is_pre_election: true
I20260809 07:48:56.166934 27034 leader_election.cc:302] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9981c18e922e4c62bd0f3fd6a22aa904, ad481fad7b3241ba9252d10843dd736d; no voters:
I20260809 07:48:56.167044 27261 raft_consensus.cc:2454] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9981c18e922e4c62bd0f3fd6a22aa904 in term 0.
I20260809 07:48:56.167436 27350 raft_consensus.cc:2781] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:56.167508 27350 raft_consensus.cc:482] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:56.167546 27350 raft_consensus.cc:3037] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.168561 27350 raft_consensus.cc:504] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.168946 27350 leader_election.cc:288] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [CANDIDATE]: Term 1 election: Requested vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67:44651)
I20260809 07:48:56.169284 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63773ed0add4b6f90985b12c195afbe" candidate_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d"
I20260809 07:48:56.169317 27261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d63773ed0add4b6f90985b12c195afbe" candidate_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b15a6406d23c4e1090d411f41bd664c2"
I20260809 07:48:56.169426 26958 raft_consensus.cc:3037] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.169461 27261 raft_consensus.cc:3037] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.170428 27261 raft_consensus.cc:2454] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9981c18e922e4c62bd0f3fd6a22aa904 in term 1.
I20260809 07:48:56.170585 26958 raft_consensus.cc:2454] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9981c18e922e4c62bd0f3fd6a22aa904 in term 1.
I20260809 07:48:56.170725 27036 leader_election.cc:302] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9981c18e922e4c62bd0f3fd6a22aa904, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:56.170915 27350 raft_consensus.cc:2781] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:56.171101 27350 raft_consensus.cc:686] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 LEADER]: Becoming Leader. State: Replica: 9981c18e922e4c62bd0f3fd6a22aa904, State: Running, Role: LEADER
I20260809 07:48:56.171360 27350 consensus_queue.cc:237] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.172675 27387 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: TxnStatusTablet state changed. Reason: New leader 9981c18e922e4c62bd0f3fd6a22aa904. Latest consensus state: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.172796 27387 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:56.173015 27390 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 07:48:56.173171 27390 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 07:48:56.174127 26790 catalog_manager.cc:5495] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } health_report { overall_health: UNKNOWN } } }
I20260809 07:48:56.191864 27379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.187604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:46599 (local address 127.0.0.1:40938)
0809 07:48:56.187773 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:56.187797 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:56.187920 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:56.188315 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:56.188318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:56.188326 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:56.188442 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.188493 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.189402 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.189406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.190170 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.190180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.190667 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.190670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.190761 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.190805 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:56.191560 (+ 755us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:56.191579 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 07:48:56.191650 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 07:48:56.191739 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 07:48:56.191883 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.187703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.18.71.66:46599)
0809 07:48:56.188030 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.188040 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.188056 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:48:56.188128 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.188130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.188178 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.188300 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:56.188611 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.189261 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.190298 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.190546 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.190654 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.190964 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:56.191459 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:56.191587 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.191661 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.191714 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":120,"threads_started":1}
I20260809 07:48:56.195091 26958 raft_consensus.cc:1261] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Refusing update from remote peer 9981c18e922e4c62bd0f3fd6a22aa904: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.195281 27261 raft_consensus.cc:1261] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Refusing update from remote peer 9981c18e922e4c62bd0f3fd6a22aa904: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.195648 27350 consensus_queue.cc:1035] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.195936 27387 consensus_queue.cc:1035] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.197177 27348 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: TxnStatusTablet state changed. Reason: New leader 9981c18e922e4c62bd0f3fd6a22aa904. Latest consensus state: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.197295 27348 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:56.197605 27362 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: TxnStatusTablet state changed. Reason: New leader 9981c18e922e4c62bd0f3fd6a22aa904. Latest consensus state: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.197723 27362 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:56.199265 27387 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.199373 27387 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:56.200018 27362 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.200141 27362 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P b15a6406d23c4e1090d411f41bd664c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:56.200098 27395 mvcc.cc:205] Tried to move back new op lower bound from 7316528071449821184 to 7316528071360671744. Current Snapshot: MvccSnapshot[applied={T|T < 7316528071449821184}]
I20260809 07:48:56.203941 27350 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.204052 27350 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P 9981c18e922e4c62bd0f3fd6a22aa904: This TxnStatusTablet replica's current role is: LEADER
I20260809 07:48:56.205129 27348 tablet_replica.cc:439] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } } }
I20260809 07:48:56.205252 27348 tablet_replica.cc:442] T d63773ed0add4b6f90985b12c195afbe P ad481fad7b3241ba9252d10843dd736d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 07:48:56.216902 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.212965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:46599 (local address 127.18.71.67:42351)
0809 07:48:56.213405 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:56.213416 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:56.213506 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:56.213797 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:56.213800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:56.213808 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:56.213930 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.213935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.214972 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.214974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.215608 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.215616 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.216214 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.216216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.216283 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.216327 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:56.216645 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:56.216652 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 07:48:56.216708 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 07:48:56.216782 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":149,"threads_started":1}
I20260809 07:48:56.216921 27394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.213044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:42351 (local address 127.18.71.66:46599)
0809 07:48:56.213153 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.213157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.213488 (+ 331us) server_negotiation.cc:407] Connection header received
0809 07:48:56.213605 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.213609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.213669 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.213750 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:56.214062 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.214863 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.215735 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.216101 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.216206 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.216474 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:56.216557 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:56.216638 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.216740 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.216793 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 07:48:56.223446 27109 raft_consensus.cc:1261] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.223459 26958 raft_consensus.cc:1261] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.223922 27349 consensus_queue.cc:1035] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.224202 27349 consensus_queue.cc:1035] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.251201 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.247714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:39425 (local address 127.18.71.67:44651)
0809 07:48:56.247845 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.247849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.247864 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:48:56.247911 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.247914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.247966 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.248047 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:56.248348 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.249170 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.250034 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.250381 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.250485 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.250755 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:56.250835 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:56.250907 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.251010 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.251065 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:48:56.251368 27412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.247287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:44651 (local address 127.18.71.66:39425)
0809 07:48:56.247617 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:56.247634 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:56.247756 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:56.248062 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:56.248065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:56.248074 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 07:48:56.248226 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.248233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.249270 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.249274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.249917 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:56.249926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.250508 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.250511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:56.250582 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.250629 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 07:48:56.250927 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 07:48:56.250934 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 07:48:56.251003 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 07:48:56.251099 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":120,"threads_started":1}
I20260809 07:48:56.281494 26789 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:56.281657 26789 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38622:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:56.283937 26789 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Sending DeleteTablet for 3 replicas of tablet 85f3de6584af4ab6955438ac6fcd146a
I20260809 07:48:56.284652 26938 tablet_service.cc:1514] Processing DeleteTablet for tablet 85f3de6584af4ab6955438ac6fcd146a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:49376
I20260809 07:48:56.284701 27238 tablet_service.cc:1514] Processing DeleteTablet for tablet 85f3de6584af4ab6955438ac6fcd146a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:45788
I20260809 07:48:56.285173 27087 tablet_service.cc:1514] Processing DeleteTablet for tablet 85f3de6584af4ab6955438ac6fcd146a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40926
I20260809 07:48:56.285498 27418 tablet_replica.cc:330] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: stopping tablet replica
I20260809 07:48:56.285656 27418 raft_consensus.cc:2229] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:56.285804 27418 raft_consensus.cc:2258] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.286520 27418 ts_tablet_manager.cc:1905] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.289356 27418 ts_tablet_manager.cc:1918] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:56.289497 27418 log.cc:1196] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/85f3de6584af4ab6955438ac6fcd146a
I20260809 07:48:56.289827 27418 ts_tablet_manager.cc:1939] T 85f3de6584af4ab6955438ac6fcd146a P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting consensus metadata
I20260809 07:48:56.290525 26773 catalog_manager.cc:4841] TS 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599): tablet 85f3de6584af4ab6955438ac6fcd146a (table test-table [id=f318863b00ed4ed9afec46ecf1b66946]) successfully deleted
I20260809 07:48:56.290912 27415 tablet_replica.cc:330] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: stopping tablet replica
I20260809 07:48:56.291072 27415 raft_consensus.cc:2229] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:56.291236 27415 raft_consensus.cc:2258] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.291356 27416 tablet_replica.cc:330] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: stopping tablet replica
I20260809 07:48:56.291489 27416 raft_consensus.cc:2229] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:56.291698 27416 raft_consensus.cc:2258] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.292320 27416 ts_tablet_manager.cc:1905] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.294030 27415 ts_tablet_manager.cc:1905] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.295351 27416 ts_tablet_manager.cc:1918] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:56.295439 27416 log.cc:1196] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/85f3de6584af4ab6955438ac6fcd146a
I20260809 07:48:56.295751 27416 ts_tablet_manager.cc:1939] T 85f3de6584af4ab6955438ac6fcd146a P b15a6406d23c4e1090d411f41bd664c2: Deleting consensus metadata
I20260809 07:48:56.296411 27415 ts_tablet_manager.cc:1918] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:56.296520 27415 log.cc:1196] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/85f3de6584af4ab6955438ac6fcd146a
I20260809 07:48:56.296525 26773 catalog_manager.cc:4841] TS b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67:44651): tablet 85f3de6584af4ab6955438ac6fcd146a (table test-table [id=f318863b00ed4ed9afec46ecf1b66946]) successfully deleted
I20260809 07:48:56.296792 27415 ts_tablet_manager.cc:1939] T 85f3de6584af4ab6955438ac6fcd146a P ad481fad7b3241ba9252d10843dd736d: Deleting consensus metadata
I20260809 07:48:56.297560 26770 catalog_manager.cc:4841] TS ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813): tablet 85f3de6584af4ab6955438ac6fcd146a (table test-table [id=f318863b00ed4ed9afec46ecf1b66946]) successfully deleted
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[26739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 07:48:56.320246 26843 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:48:56.313335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.18.71.126:33409)
0809 07:48:56.313468 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.313472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.313717 (+ 245us) server_negotiation.cc:407] Connection header received
0809 07:48:56.313814 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.313817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.313890 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.313988 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:56.314712 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.315510 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.316359 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.316736 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.316852 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.320108 (+ 3256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38632: BlockingRecv error: recv got EOF from 127.0.0.1:38632 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[26739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM
I20260809 07:48:56.330667 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.321967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.18.71.126:33409)
0809 07:48:56.322084 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.322088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.322183 (+ 95us) server_negotiation.cc:407] Connection header received
0809 07:48:56.322264 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.322267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.322318 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.322412 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:56.322929 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.323681 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.324431 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.324784 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.324905 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.327395 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:56.327400 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:56.327403 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:56.327429 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:56.328996 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:56.329419 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:56.329422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:56.329424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:56.329507 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:56.329801 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:56.329804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:56.329806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:56.329983 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:56.330117 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.330353 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.330470 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:48:56.333480 26789 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:48:56.334023 26789 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:56.339022 26938 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb0b380a5e14a808954bed165e864d1 (DEFAULT_TABLE table=test-table [id=34d5476ee3f04f2983493e004bdc55ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:56.339391 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb0b380a5e14a808954bed165e864d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.339363 27087 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb0b380a5e14a808954bed165e864d1 (DEFAULT_TABLE table=test-table [id=34d5476ee3f04f2983493e004bdc55ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:56.339540 27238 tablet_service.cc:1467] Processing CreateTablet for tablet 7eb0b380a5e14a808954bed165e864d1 (DEFAULT_TABLE table=test-table [id=34d5476ee3f04f2983493e004bdc55ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:56.339643 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb0b380a5e14a808954bed165e864d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.339823 27238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eb0b380a5e14a808954bed165e864d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.342160 27341 tablet_bootstrap.cc:492] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: Bootstrap starting.
I20260809 07:48:56.342176 27344 tablet_bootstrap.cc:492] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: Bootstrap starting.
I20260809 07:48:56.342239 27342 tablet_bootstrap.cc:492] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: Bootstrap starting.
I20260809 07:48:56.343127 27341 tablet_bootstrap.cc:654] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.343127 27342 tablet_bootstrap.cc:654] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.343147 27344 tablet_bootstrap.cc:654] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.344575 27342 tablet_bootstrap.cc:492] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: No bootstrap required, opened a new log
I20260809 07:48:56.344681 27342 ts_tablet_manager.cc:1397] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 07:48:56.345000 27341 tablet_bootstrap.cc:492] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: No bootstrap required, opened a new log
I20260809 07:48:56.345088 27341 ts_tablet_manager.cc:1397] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:48:56.345089 27342 raft_consensus.cc:348] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.345206 27342 raft_consensus.cc:374] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.345237 27342 raft_consensus.cc:729] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.345360 27342 consensus_queue.cc:260] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.345606 27342 ts_tablet_manager.cc:1428] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:56.345580 27341 raft_consensus.cc:348] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.345690 27341 raft_consensus.cc:374] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.345697 27344 tablet_bootstrap.cc:492] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: No bootstrap required, opened a new log
I20260809 07:48:56.345731 27341 raft_consensus.cc:729] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad481fad7b3241ba9252d10843dd736d, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.345778 27344 ts_tablet_manager.cc:1397] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260809 07:48:56.345870 27341 consensus_queue.cc:260] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.346139 27341 ts_tablet_manager.cc:1428] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:56.346158 27344 raft_consensus.cc:348] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.346264 27344 raft_consensus.cc:374] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.346306 27344 raft_consensus.cc:729] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9981c18e922e4c62bd0f3fd6a22aa904, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.346439 27344 consensus_queue.cc:260] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.346736 27344 ts_tablet_manager.cc:1428] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:56.529443 27362 raft_consensus.cc:482] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:56.529584 27362 raft_consensus.cc:504] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.530015 27362 leader_election.cc:288] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:56.530501 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb0b380a5e14a808954bed165e864d1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" is_pre_election: true
I20260809 07:48:56.530558 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb0b380a5e14a808954bed165e864d1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d" is_pre_election: true
I20260809 07:48:56.530678 27109 raft_consensus.cc:2454] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:56.530719 26958 raft_consensus.cc:2454] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:56.530987 27188 leader_election.cc:302] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9981c18e922e4c62bd0f3fd6a22aa904, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:56.531199 27362 raft_consensus.cc:2781] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:56.531262 27362 raft_consensus.cc:482] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:56.531291 27362 raft_consensus.cc:3037] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.532691 27362 raft_consensus.cc:504] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.533103 27362 leader_election.cc:288] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Requested vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:56.533396 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb0b380a5e14a808954bed165e864d1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d"
I20260809 07:48:56.533519 26958 raft_consensus.cc:3037] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.533475 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eb0b380a5e14a808954bed165e864d1" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904"
I20260809 07:48:56.533579 27109 raft_consensus.cc:3037] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.534339 26958 raft_consensus.cc:2454] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:56.534610 27185 leader_election.cc:302] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad481fad7b3241ba9252d10843dd736d, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:56.534673 27109 raft_consensus.cc:2454] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:56.534850 27362 raft_consensus.cc:2781] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:56.534950 27362 raft_consensus.cc:686] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Becoming Leader. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Running, Role: LEADER
I20260809 07:48:56.535082 27362 consensus_queue.cc:237] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.536926 26788 catalog_manager.cc:5495] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "b15a6406d23c4e1090d411f41bd664c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } health_report { overall_health: HEALTHY } } }
I20260809 07:48:56.585204 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.582101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44946 (local address 127.18.71.67:44651)
0809 07:48:56.582239 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:56.582242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:56.582278 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:48:56.582329 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:56.582331 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:56.582375 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:56.582431 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:48:56.582729 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.583280 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.584127 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:56.584351 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:56.584436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:56.584775 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:48:56.584860 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:48:56.584933 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:56.585040 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 07:48:56.585091 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 07:48:56.587364 26958 raft_consensus.cc:1261] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.587366 27109 raft_consensus.cc:1261] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.587805 27413 consensus_queue.cc:1035] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.587944 27362 consensus_queue.cc:1035] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.595925 26788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:56.596107 26788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:56.598104 26788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Sending DeleteTablet for 3 replicas of tablet 7eb0b380a5e14a808954bed165e864d1
I20260809 07:48:56.598665 26938 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb0b380a5e14a808954bed165e864d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:49376
I20260809 07:48:56.598768 27087 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb0b380a5e14a808954bed165e864d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40926
I20260809 07:48:56.598846 27415 tablet_replica.cc:330] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: stopping tablet replica
I20260809 07:48:56.598965 27418 tablet_replica.cc:330] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: stopping tablet replica
I20260809 07:48:56.598953 27415 raft_consensus.cc:2229] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:56.599104 27415 raft_consensus.cc:2258] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.599094 27418 raft_consensus.cc:2229] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:56.599247 27418 raft_consensus.cc:2258] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.599287 27238 tablet_service.cc:1514] Processing DeleteTablet for tablet 7eb0b380a5e14a808954bed165e864d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:45788
I20260809 07:48:56.599548 27416 tablet_replica.cc:330] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: stopping tablet replica
I20260809 07:48:56.599692 27416 raft_consensus.cc:2229] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:56.599849 27416 raft_consensus.cc:2258] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.599779 26788 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 07:48:56.600044 27415 ts_tablet_manager.cc:1905] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 07:48:56.600293 26788 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:48:56.601018 27416 ts_tablet_manager.cc:1905] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.601017 27418 ts_tablet_manager.cc:1905] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.603137 27415 ts_tablet_manager.cc:1918] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:56.603217 27415 log.cc:1196] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/7eb0b380a5e14a808954bed165e864d1
I20260809 07:48:56.603538 27416 ts_tablet_manager.cc:1918] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:56.603562 27415 ts_tablet_manager.cc:1939] T 7eb0b380a5e14a808954bed165e864d1 P ad481fad7b3241ba9252d10843dd736d: Deleting consensus metadata
I20260809 07:48:56.603613 27416 log.cc:1196] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/7eb0b380a5e14a808954bed165e864d1
I20260809 07:48:56.603874 27416 ts_tablet_manager.cc:1939] T 7eb0b380a5e14a808954bed165e864d1 P b15a6406d23c4e1090d411f41bd664c2: Deleting consensus metadata
I20260809 07:48:56.604236 26770 catalog_manager.cc:4841] TS ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813): tablet 7eb0b380a5e14a808954bed165e864d1 (table test-table [id=34d5476ee3f04f2983493e004bdc55ef]) successfully deleted
I20260809 07:48:56.604509 26773 catalog_manager.cc:4841] TS b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67:44651): tablet 7eb0b380a5e14a808954bed165e864d1 (table test-table [id=34d5476ee3f04f2983493e004bdc55ef]) successfully deleted
I20260809 07:48:56.604882 27418 ts_tablet_manager.cc:1918] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:48:56.604938 27418 log.cc:1196] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/7eb0b380a5e14a808954bed165e864d1
I20260809 07:48:56.605217 27418 ts_tablet_manager.cc:1939] T 7eb0b380a5e14a808954bed165e864d1 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting consensus metadata
I20260809 07:48:56.605932 26773 catalog_manager.cc:4841] TS 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599): tablet 7eb0b380a5e14a808954bed165e864d1 (table test-table [id=34d5476ee3f04f2983493e004bdc55ef]) successfully deleted
I20260809 07:48:56.606338 26938 tablet_service.cc:1467] Processing CreateTablet for tablet 772a41bcb2484ab3860afb7424689741 (DEFAULT_TABLE table=test-table [id=e84304adb6914b1dac3838a6fdda83f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:56.606504 27087 tablet_service.cc:1467] Processing CreateTablet for tablet 772a41bcb2484ab3860afb7424689741 (DEFAULT_TABLE table=test-table [id=e84304adb6914b1dac3838a6fdda83f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:56.606618 26938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772a41bcb2484ab3860afb7424689741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.606714 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772a41bcb2484ab3860afb7424689741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.608624 27341 tablet_bootstrap.cc:492] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d: Bootstrap starting.
I20260809 07:48:56.608772 27344 tablet_bootstrap.cc:492] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: Bootstrap starting.
I20260809 07:48:56.609622 27344 tablet_bootstrap.cc:654] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.609939 27341 tablet_bootstrap.cc:654] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.610317 27238 tablet_service.cc:1467] Processing CreateTablet for tablet 772a41bcb2484ab3860afb7424689741 (DEFAULT_TABLE table=test-table [id=e84304adb6914b1dac3838a6fdda83f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:48:56.610584 27238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 772a41bcb2484ab3860afb7424689741. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:56.612411 27342 tablet_bootstrap.cc:492] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: Bootstrap starting.
I20260809 07:48:56.612941 27344 tablet_bootstrap.cc:492] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: No bootstrap required, opened a new log
I20260809 07:48:56.613049 27344 ts_tablet_manager.cc:1397] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 07:48:56.613273 27342 tablet_bootstrap.cc:654] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:56.613458 27344 raft_consensus.cc:348] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.613574 27344 raft_consensus.cc:374] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.613618 27344 raft_consensus.cc:729] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9981c18e922e4c62bd0f3fd6a22aa904, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.613749 27344 consensus_queue.cc:260] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.614051 27344 ts_tablet_manager.cc:1428] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:56.614594 27342 tablet_bootstrap.cc:492] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: No bootstrap required, opened a new log
I20260809 07:48:56.614708 27342 ts_tablet_manager.cc:1397] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 07:48:56.615043 27341 tablet_bootstrap.cc:492] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d: No bootstrap required, opened a new log
I20260809 07:48:56.615044 27342 raft_consensus.cc:348] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.615137 27341 ts_tablet_manager.cc:1397] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 07:48:56.615170 27342 raft_consensus.cc:374] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.615216 27342 raft_consensus.cc:729] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.615351 27342 consensus_queue.cc:260] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.615631 27342 ts_tablet_manager.cc:1428] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 07:48:56.615617 27341 raft_consensus.cc:348] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.615726 27341 raft_consensus.cc:374] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:56.615774 27341 raft_consensus.cc:729] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad481fad7b3241ba9252d10843dd736d, State: Initialized, Role: FOLLOWER
I20260809 07:48:56.615900 27341 consensus_queue.cc:260] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.616425 27341 ts_tablet_manager.cc:1428] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:48:56.829064 27362 raft_consensus.cc:482] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:48:56.829241 27362 raft_consensus.cc:504] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.829784 27362 leader_election.cc:288] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:56.830348 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772a41bcb2484ab3860afb7424689741" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" is_pre_election: true
I20260809 07:48:56.830349 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772a41bcb2484ab3860afb7424689741" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d" is_pre_election: true
I20260809 07:48:56.830519 27109 raft_consensus.cc:2454] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:56.830519 26958 raft_consensus.cc:2454] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 0.
I20260809 07:48:56.830896 27188 leader_election.cc:302] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9981c18e922e4c62bd0f3fd6a22aa904, ad481fad7b3241ba9252d10843dd736d, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:56.831107 27362 raft_consensus.cc:2781] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:48:56.831197 27362 raft_consensus.cc:482] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:48:56.831252 27362 raft_consensus.cc:3037] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.832304 27362 raft_consensus.cc:504] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.832705 27362 leader_election.cc:288] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Requested vote from peers ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813), 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599)
I20260809 07:48:56.833105 26958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772a41bcb2484ab3860afb7424689741" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad481fad7b3241ba9252d10843dd736d"
I20260809 07:48:56.833129 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "772a41bcb2484ab3860afb7424689741" candidate_uuid: "b15a6406d23c4e1090d411f41bd664c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9981c18e922e4c62bd0f3fd6a22aa904"
I20260809 07:48:56.833253 27109 raft_consensus.cc:3037] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.833253 26958 raft_consensus.cc:3037] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:56.834410 27109 raft_consensus.cc:2454] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:56.834410 26958 raft_consensus.cc:2454] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b15a6406d23c4e1090d411f41bd664c2 in term 1.
I20260809 07:48:56.834734 27188 leader_election.cc:302] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9981c18e922e4c62bd0f3fd6a22aa904, b15a6406d23c4e1090d411f41bd664c2; no voters:
I20260809 07:48:56.834955 27362 raft_consensus.cc:2781] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:56.835052 27362 raft_consensus.cc:686] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Becoming Leader. State: Replica: b15a6406d23c4e1090d411f41bd664c2, State: Running, Role: LEADER
I20260809 07:48:56.835202 27362 consensus_queue.cc:237] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } }
I20260809 07:48:56.836831 26789 catalog_manager.cc:5495] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "b15a6406d23c4e1090d411f41bd664c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b15a6406d23c4e1090d411f41bd664c2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 44651 } health_report { overall_health: HEALTHY } } }
I20260809 07:48:56.856386 27109 raft_consensus.cc:1261] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.856510 26958 raft_consensus.cc:1261] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Refusing update from remote peer b15a6406d23c4e1090d411f41bd664c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:48:56.856966 27413 consensus_queue.cc:1035] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9981c18e922e4c62bd0f3fd6a22aa904" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 46599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.857136 27362 consensus_queue.cc:1035] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad481fad7b3241ba9252d10843dd736d" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 38813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:48:56.887494 26788 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:56.887692 26788 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:48:56.889724 26788 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 224e4ec19bf24f82b127e966bf650b77: Sending DeleteTablet for 3 replicas of tablet 772a41bcb2484ab3860afb7424689741
I20260809 07:48:56.890328 26938 tablet_service.cc:1514] Processing DeleteTablet for tablet 772a41bcb2484ab3860afb7424689741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:49376
I20260809 07:48:56.890358 27087 tablet_service.cc:1514] Processing DeleteTablet for tablet 772a41bcb2484ab3860afb7424689741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40926
I20260809 07:48:56.890547 27418 tablet_replica.cc:330] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: stopping tablet replica
I20260809 07:48:56.890553 27415 tablet_replica.cc:330] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d: stopping tablet replica
I20260809 07:48:56.890552 27238 tablet_service.cc:1514] Processing DeleteTablet for tablet 772a41bcb2484ab3860afb7424689741 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:48:56 UTC) from {username='oryx', principal='oryx/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:45788
I20260809 07:48:56.890684 27418 raft_consensus.cc:2229] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:56.890683 27415 raft_consensus.cc:2229] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:48:56.890712 27416 tablet_replica.cc:330] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: stopping tablet replica
I20260809 07:48:56.890805 27415 raft_consensus.cc:2258] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.890823 27418 raft_consensus.cc:2258] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.890831 27416 raft_consensus.cc:2229] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:48:56.891017 27416 raft_consensus.cc:2258] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:48:56.891654 27415 ts_tablet_manager.cc:1905] T 772a41bcb2484ab3860afb7424689741 P ad481fad7b3241ba9252d10843dd736d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.891777 27416 ts_tablet_manager.cc:1905] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.891777 27418 ts_tablet_manager.cc:1905] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:48:56.892002 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 26869
I20260809 07:48:56.894275 27416 ts_tablet_manager.cc:1918] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:56.894275 27418 ts_tablet_manager.cc:1918] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 07:48:56.894362 27416 log.cc:1196] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/772a41bcb2484ab3860afb7424689741
I20260809 07:48:56.894362 27418 log.cc:1196] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/772a41bcb2484ab3860afb7424689741
I20260809 07:48:56.894652 27416 ts_tablet_manager.cc:1939] T 772a41bcb2484ab3860afb7424689741 P b15a6406d23c4e1090d411f41bd664c2: Deleting consensus metadata
I20260809 07:48:56.894652 27418 ts_tablet_manager.cc:1939] T 772a41bcb2484ab3860afb7424689741 P 9981c18e922e4c62bd0f3fd6a22aa904: Deleting consensus metadata
I20260809 07:48:56.895332 26773 catalog_manager.cc:4841] TS b15a6406d23c4e1090d411f41bd664c2 (127.18.71.67:44651): tablet 772a41bcb2484ab3860afb7424689741 (table test-table [id=e84304adb6914b1dac3838a6fdda83f4]) successfully deleted
I20260809 07:48:56.895511 26773 catalog_manager.cc:4841] TS 9981c18e922e4c62bd0f3fd6a22aa904 (127.18.71.66:46599): tablet 772a41bcb2484ab3860afb7424689741 (table test-table [id=e84304adb6914b1dac3838a6fdda83f4]) successfully deleted
I20260809 07:48:56.902750 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 27017
W20260809 07:48:56.902736 26770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:38813 (error 108)
I20260809 07:48:56.905661 27450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:56.904820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:38813 (local address 127.0.0.1:53092)
0809 07:48:56.905444 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:56.905529 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:38813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":63,"threads_started":1}
W20260809 07:48:56.905882 26770 catalog_manager.cc:4584] TS ad481fad7b3241ba9252d10843dd736d (127.18.71.65:38813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 772a41bcb2484ab3860afb7424689741: Network error: Client connection negotiation failed: client connection to 127.18.71.65:38813: connect: Connection refused (error 111)
I20260809 07:48:56.912487 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 27168
I20260809 07:48:56.920707 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 26753
2026-08-09T07:48:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3598 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[27453](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[27453](info): set up 2 sockets
Aug 09 07:48:56 dist-test-slave-10pc krb5kdc[27453](info): commencing operation
krb5kdc: starting...
W20260809 07:48:58.016412 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:58 dist-test-slave-10pc krb5kdc[27453](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:58.147511 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46213
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:46213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:58.267220 27466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:58.267509 27466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:58.267581 27466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:58.271643 27466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:58.271731 27466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:58.271770 27466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:58.271829 27466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:58.271867 27466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:58.277648 27466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:46213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:58.278733 27466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:58.279763 27466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:58.288357 27475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:58.288357 27472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:58.288384 27471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:58.288678 27466 server_base.cc:1034] running on GCE node
I20260809 07:48:58.289328 27466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:58.290360 27466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:58.291558 27466 hybrid_clock.cc:648] HybridClock initialized: now 1786261738291536 us; error 43 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T07:48:58Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1382 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:48:58 dist-test-slave-10pc krb5kdc[27480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:48:58 dist-test-slave-10pc krb5kdc[27480](info): set up 2 sockets
Aug 09 07:48:58 dist-test-slave-10pc krb5kdc[27480](info): commencing operation
krb5kdc: starting...
W20260809 07:48:59.381969 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:48:59 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:48:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:48:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:48:59.511497 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:34265
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:34265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:59.623761 27493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:59.624091 27493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:59.624191 27493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:59.628777 27493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:48:59.628859 27493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:59.628897 27493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:59.628935 27493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:48:59.628970 27493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:48:59.634184 27493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:34265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:34265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:59.635260 27493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:59.636298 27493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:59.644872 27500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:59.644912 27499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:59.644980 27493 server_base.cc:1034] running on GCE node
W20260809 07:48:59.644912 27502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:59.645677 27493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:59.646780 27493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:59.647964 27493 hybrid_clock.cc:648] HybridClock initialized: now 1786261739647952 us; error 25 us; skew 500 ppm
Aug 09 07:48:59 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261739, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:59.650857 27493 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:48:59.651768 27493 webserver.cc:460] Webserver started at http://127.18.71.126:34031/ using document root <none> and password file <none>
I20260809 07:48:59.652333 27493 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:59.652423 27493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:59.652700 27493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:59.654744 27493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "d2475b56060e4d199613d36a6f51a867"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "944b5c5747aae188c91dc9cdbcc0ca01"
server_key_iv: "66f6bf83e7b0f91a768bccc642560684"
server_key_version: "encryptionkey@0"
I20260809 07:48:59.655329 27493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "d2475b56060e4d199613d36a6f51a867"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "944b5c5747aae188c91dc9cdbcc0ca01"
server_key_iv: "66f6bf83e7b0f91a768bccc642560684"
server_key_version: "encryptionkey@0"
I20260809 07:48:59.659015 27493 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:48:59.661399 27510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.662598 27493 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:48:59.662745 27493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "d2475b56060e4d199613d36a6f51a867"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "944b5c5747aae188c91dc9cdbcc0ca01"
server_key_iv: "66f6bf83e7b0f91a768bccc642560684"
server_key_version: "encryptionkey@0"
I20260809 07:48:59.662853 27493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:59.675602 27493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:59.678571 27493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:59.678783 27493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:59.685660 27493 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:34265
I20260809 07:48:59.685685 27567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:34265 every 8 connection(s)
I20260809 07:48:59.686553 27493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:48:59.688087 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 27493
I20260809 07:48:59.688215 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:48:59.688534 18717 external_mini_cluster.cc:1428] Setting key be61767d6d80cba2e337e3e796eae02b
I20260809 07:48:59.689587 27568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:48:59.695180 27568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Bootstrap starting.
Aug 09 07:48:59 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:48:59.697487 27568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Neither blocks nor log segments found. Creating new log.
I20260809 07:48:59.698320 27568 log.cc:824] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Log is configured to *not* fsync() on all Append() calls
I20260809 07:48:59.700488 27568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: No bootstrap required, opened a new log
I20260809 07:48:59.702704 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:59.689942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.18.71.126:34265)
0809 07:48:59.690370 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:59.690385 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:59.690419 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:48:59.691043 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:59.691058 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:59.691369 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:59.691692 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:59.692575 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:59.693592 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:59.694511 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:59.694984 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:59.695167 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:59.697669 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:59.697675 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:59.697697 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:59.697723 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:59.699629 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:59.700086 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:59.700091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:59.700097 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:59.700240 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:59.700528 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:59.700534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:59.700535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:59.701021 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:59.701204 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:59.701454 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 07:48:59.701748 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":148,"threads_started":1}
I20260809 07:48:59.704830 27568 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:48:59.705116 27568 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:48:59.705158 27568 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2475b56060e4d199613d36a6f51a867, State: Initialized, Role: FOLLOWER
I20260809 07:48:59.705505 27568 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:48:59.705613 27568 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:48:59.705658 27568 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:48:59.705726 27568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:48:59.706580 27568 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:48:59.706923 27568 leader_election.cc:302] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2475b56060e4d199613d36a6f51a867; no voters:
I20260809 07:48:59.707216 27568 leader_election.cc:288] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:48:59.707335 27573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:48:59.707556 27573 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 LEADER]: Becoming Leader. State: Replica: d2475b56060e4d199613d36a6f51a867, State: Running, Role: LEADER
I20260809 07:48:59.707872 27573 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:48:59.708436 27568 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:48:59.709553 27574 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2475b56060e4d199613d36a6f51a867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } } }
I20260809 07:48:59.709679 27574 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:59.709965 27575 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2475b56060e4d199613d36a6f51a867. Latest consensus state: current_term: 1 leader_uuid: "d2475b56060e4d199613d36a6f51a867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } } }
I20260809 07:48:59.710064 27575 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: This master's current role is: LEADER
I20260809 07:48:59.710397 27584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:48:59.713191 27584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:48:59.718140 27584 catalog_manager.cc:1347] Generated new cluster ID: 57c6eecc50a94eb099bad52872af48ed
I20260809 07:48:59.718204 27584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:48:59.725667 27584 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:48:59.726809 27584 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:48:59.733623 27584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Generated new TSK 0
I20260809 07:48:59.734339 27584 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:48:59.767207 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:34265
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:48:59.880918 27596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:48:59.881186 27596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:48:59.881254 27596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:48:59.885200 27596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:48:59.885270 27596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:48:59.885349 27596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:48:59.890380 27596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:34265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:48:59.891402 27596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:48:59.892452 27596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:48:59.901674 27601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:59.901686 27605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:48:59.901778 27602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:48:59.901822 27596 server_base.cc:1034] running on GCE node
I20260809 07:48:59.902384 27596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:48:59.902984 27596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:48:59.904155 27596 hybrid_clock.cc:648] HybridClock initialized: now 1786261739904113 us; error 51 us; skew 500 ppm
Aug 09 07:48:59 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261739, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:48:59.907065 27596 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:48:59.908001 27596 webserver.cc:460] Webserver started at http://127.18.71.65:42187/ using document root <none> and password file <none>
I20260809 07:48:59.908617 27596 fs_manager.cc:359] Metadata directory not provided
I20260809 07:48:59.908708 27596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:48:59.908948 27596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:48:59.911069 27596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "9be0183a82a94520b3c6092f4ca551b2"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "e8b8e400d58bc34ab9f24c2d034c266f"
server_key_iv: "9bb5479a4ed2e34fcd9729773c717f4a"
server_key_version: "encryptionkey@0"
I20260809 07:48:59.911652 27596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "9be0183a82a94520b3c6092f4ca551b2"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "e8b8e400d58bc34ab9f24c2d034c266f"
server_key_iv: "9bb5479a4ed2e34fcd9729773c717f4a"
server_key_version: "encryptionkey@0"
I20260809 07:48:59.915341 27596 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 07:48:59.917842 27612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.919019 27596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:48:59.919157 27596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "9be0183a82a94520b3c6092f4ca551b2"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "e8b8e400d58bc34ab9f24c2d034c266f"
server_key_iv: "9bb5479a4ed2e34fcd9729773c717f4a"
server_key_version: "encryptionkey@0"
I20260809 07:48:59.919262 27596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:48:59.956442 27596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:48:59.960216 27596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:48:59.960508 27596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:48:59.961184 27596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:48:59.962126 27596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:48:59.962203 27596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.962275 27596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:48:59.962317 27596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:48:59.971621 27596 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:35851
I20260809 07:48:59.971812 27736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:35851 every 8 connection(s)
I20260809 07:48:59.972661 27596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:48:59.974623 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 27596
I20260809 07:48:59.974754 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:48:59.975100 18717 external_mini_cluster.cc:1428] Setting key c292ce2affa1e96093d8660729660c45
Aug 09 07:48:59 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261739, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:48:59.988632 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:59.974656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:39073 (local address 127.18.71.126:34265)
0809 07:48:59.974850 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 07:48:59.974855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:48:59.976167 (+ 1312us) server_negotiation.cc:407] Connection header received
0809 07:48:59.977616 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:48:59.977622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:48:59.977686 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:48:59.977779 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:48:59.979127 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:59.979667 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:59.980912 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:48:59.981163 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:48:59.981274 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:59.984423 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:48:59.984427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:48:59.984430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:48:59.984519 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:48:59.986112 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:59.986912 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:59.986919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:59.986920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:59.987021 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:48:59.987383 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:48:59.987386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:48:59.987388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:48:59.987634 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:48:59.987744 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:48:59.988258 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 07:48:59.988397 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
I20260809 07:48:59.989562 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:48:59.975169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:39073)
0809 07:48:59.975791 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 07:48:59.975836 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 07:48:59.977383 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:48:59.977942 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:48:59.977949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:48:59.978433 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:48:59.978977 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:59.978992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:59.979798 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:59.979802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:59.980785 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:48:59.980795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:48:59.981314 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:48:59.981318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:48:59.981501 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:48:59.982226 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:48:59.982242 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:48:59.984228 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:48:59.986259 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:59.986265 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:59.986282 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:59.986771 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:59.987121 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:48:59.987124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:48:59.987126 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:48:59.987280 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:48:59.987744 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:48:59.987752 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:48:59.988016 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 07:48:59.988215 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 07:48:59.988579 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":153,"threads_started":1}
I20260809 07:48:59.990880 27738 heartbeater.cc:347] Connected to a master server at 127.18.71.126:34265
I20260809 07:48:59.991154 27738 heartbeater.cc:464] Registering TS with master...
I20260809 07:48:59.991730 27738 heartbeater.cc:511] Master 127.18.71.126:34265 requested a full tablet report, sending...
I20260809 07:48:59.993338 27528 ts_manager.cc:194] Registered new tserver with Master: 9be0183a82a94520b3c6092f4ca551b2 (127.18.71.65:35851)
I20260809 07:48:59.995024 27528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:39073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:49:00.009071 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:34265
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:49:00.123907 27743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:00.124163 27743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:00.124207 27743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:00.128161 27743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:00.128214 27743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:00.128270 27743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:49:00.133404 27743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:34265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:00.134349 27743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:00.135347 27743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:00.145063 27752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.145155 27749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.145268 27743 server_base.cc:1034] running on GCE node
W20260809 07:49:00.145155 27748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.145776 27743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:00.146427 27743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:00.147593 27743 hybrid_clock.cc:648] HybridClock initialized: now 1786261740147578 us; error 25 us; skew 500 ppm
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:00.150408 27743 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:49:00.151315 27743 webserver.cc:460] Webserver started at http://127.18.71.66:33857/ using document root <none> and password file <none>
I20260809 07:49:00.151891 27743 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:00.151952 27743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:00.152127 27743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:00.154173 27743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "5fcfb5760d0e40c394c30f6837107238"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "064ac6fd2a1fd29a719c34db9ac6512e"
server_key_iv: "26eaa118ad18f76cf616279c5dfc1f81"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.154695 27743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "5fcfb5760d0e40c394c30f6837107238"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "064ac6fd2a1fd29a719c34db9ac6512e"
server_key_iv: "26eaa118ad18f76cf616279c5dfc1f81"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.158377 27743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:49:00.160676 27759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.161939 27743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:49:00.162077 27743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "5fcfb5760d0e40c394c30f6837107238"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "064ac6fd2a1fd29a719c34db9ac6512e"
server_key_iv: "26eaa118ad18f76cf616279c5dfc1f81"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.162191 27743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:00.179312 27743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:00.182788 27743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:00.183000 27743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:00.183638 27743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:00.184571 27743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:00.184620 27743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.184686 27743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:00.184722 27743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.193845 27743 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:40937
I20260809 07:49:00.193878 27879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:40937 every 8 connection(s)
I20260809 07:49:00.194725 27743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:00.196549 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 27743
I20260809 07:49:00.196692 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:49:00.197021 18717 external_mini_cluster.cc:1428] Setting key 2c60ecd70035f8b05bb61ef1b0ec7b04
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:49:00.209201 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:00.196559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:52637 (local address 127.18.71.126:34265)
0809 07:49:00.197531 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:00.197537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:00.197791 (+ 254us) server_negotiation.cc:407] Connection header received
0809 07:49:00.199182 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:00.199186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:00.199239 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:00.199323 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:00.200511 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.201044 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.202523 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.202770 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.202886 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.205422 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:00.205425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:00.205427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:00.205450 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:00.206957 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:00.207514 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:00.207517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:00.207518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:00.207599 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:00.208008 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:00.208015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:00.208017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:00.208238 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:00.208351 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:00.208906 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 07:49:00.209027 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:49:00.210156 27882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:00.197062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:52637)
0809 07:49:00.197627 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:00.197692 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:00.198956 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:00.199510 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:00.199517 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:00.199857 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:00.200339 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.200353 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.201174 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.201178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.202397 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.202407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.202925 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.202930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.203101 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.203701 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:00.203718 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:00.205248 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:00.207118 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:00.207123 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:00.207138 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:00.207393 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:00.207691 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:00.207693 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:00.207694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:00.207826 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:00.208351 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:00.208359 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:00.208668 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 07:49:00.208859 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 07:49:00.209205 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":145,"threads_started":1}
I20260809 07:49:00.211445 27880 heartbeater.cc:347] Connected to a master server at 127.18.71.126:34265
I20260809 07:49:00.211725 27880 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:00.212221 27880 heartbeater.cc:511] Master 127.18.71.126:34265 requested a full tablet report, sending...
I20260809 07:49:00.213367 27528 ts_manager.cc:194] Registered new tserver with Master: 5fcfb5760d0e40c394c30f6837107238 (127.18.71.66:40937)
I20260809 07:49:00.214232 27528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:52637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:49:00.226651 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:34265
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:49:00.342813 27886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:00.343103 27886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:00.343147 27886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:00.347280 27886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:00.347345 27886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:00.347429 27886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:49:00.352825 27886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:34265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:00.353933 27886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:00.355003 27886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:00.365120 27893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.365087 27894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.365120 27896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.365227 27886 server_base.cc:1034] running on GCE node
I20260809 07:49:00.365733 27886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:00.366499 27886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:00.367662 27886 hybrid_clock.cc:648] HybridClock initialized: now 1786261740367642 us; error 30 us; skew 500 ppm
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:00.370534 27886 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:49:00.371335 27886 webserver.cc:460] Webserver started at http://127.18.71.67:45221/ using document root <none> and password file <none>
I20260809 07:49:00.371847 27886 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:00.371903 27886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:00.372083 27886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:00.374200 27886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "2cd24f66284f42a8b2f947409527661f"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "19899b4c7918f980baafbc795886604a"
server_key_iv: "0887afea4b68c405422678aefed47675"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.374725 27886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd24f66284f42a8b2f947409527661f"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "19899b4c7918f980baafbc795886604a"
server_key_iv: "0887afea4b68c405422678aefed47675"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.378394 27886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:49:00.380721 27903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.381918 27886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:49:00.382025 27886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "2cd24f66284f42a8b2f947409527661f"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "19899b4c7918f980baafbc795886604a"
server_key_iv: "0887afea4b68c405422678aefed47675"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.382131 27886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:00.408030 27886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:00.411631 27886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:00.411782 27886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:00.412396 27886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:00.413559 27886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:00.413628 27886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.413704 27886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:00.413743 27886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.426127 27886 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:37655
I20260809 07:49:00.426151 28028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:37655 every 8 connection(s)
I20260809 07:49:00.427078 27886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:00.433867 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 27886
I20260809 07:49:00.434039 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:00.434321 18717 external_mini_cluster.cc:1428] Setting key 33a3b1665332d3aa9085965372ac4a60
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:49:00.440799 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:00.429084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:59545 (local address 127.18.71.126:34265)
0809 07:49:00.429227 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:00.429231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:00.430054 (+ 823us) server_negotiation.cc:407] Connection header received
0809 07:49:00.431479 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:00.431483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:00.431540 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:00.431654 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:00.432895 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.433426 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.434714 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.434933 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.435042 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.437512 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:00.437515 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:00.437517 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:00.437552 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:00.439024 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:00.439465 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:00.439468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:00.439469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:00.439556 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:00.439851 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:00.439853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:00.439854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:00.440020 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:00.440096 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:00.440586 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 07:49:00.440685 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 07:49:00.441602 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:00.429450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:59545)
0809 07:49:00.429904 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:00.429949 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:00.431262 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:00.431832 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:00.431839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:00.432203 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:00.432748 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.432763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.433557 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.433565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.434584 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.434594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.435084 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.435088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.435270 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.435856 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:00.435872 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:00.437364 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:00.439125 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:00.439129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:00.439141 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:00.439381 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:00.439644 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:00.439646 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:00.439647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:00.439773 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:00.440109 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:00.440115 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:00.440331 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 07:49:00.440551 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 07:49:00.440808 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":113,"threads_started":1}
I20260809 07:49:00.442783 28029 heartbeater.cc:347] Connected to a master server at 127.18.71.126:34265
I20260809 07:49:00.442982 28029 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:00.443409 28029 heartbeater.cc:511] Master 127.18.71.126:34265 requested a full tablet report, sending...
I20260809 07:49:00.444365 27528 ts_manager.cc:194] Registered new tserver with Master: 2cd24f66284f42a8b2f947409527661f (127.18.71.67:37655)
I20260809 07:49:00.444965 27528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:59545
I20260809 07:49:00.448002 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab.
Entry for principal oryx/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab.
WARNING: no policy specified for oryx/127.18.71.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab.
Entry for principal oryx/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab.
WARNING: no policy specified for oryx/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab.
Entry for principal oryx/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab.
I20260809 07:49:00.519124 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 27596
I20260809 07:49:00.527243 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 27743
I20260809 07:49:00.534386 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 27886
I20260809 07:49:00.541891 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 27493
I20260809 07:49:00.549976 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:34265
--webserver_interface=127.18.71.126
--webserver_port=34031
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:34265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:49:00.664036 28040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:00.664309 28040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:00.664348 28040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:00.668373 28040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:00.668434 28040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:00.668468 28040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:00.668490 28040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:00.668505 28040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:00.673733 28040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:34265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:34265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=34031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:00.674805 28040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:00.675846 28040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:00.684118 28047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.684108 28049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.684106 28046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.684536 28040 server_base.cc:1034] running on GCE node
I20260809 07:49:00.685110 28040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:00.686239 28040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:00.687427 28040 hybrid_clock.cc:648] HybridClock initialized: now 1786261740687418 us; error 31 us; skew 500 ppm
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261740, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:00.690253 28040 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:49:00.691098 28040 webserver.cc:460] Webserver started at http://127.18.71.126:34031/ using document root <none> and password file <none>
I20260809 07:49:00.691663 28040 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:00.691774 28040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:00.695515 28040 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:49:00.697705 28056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.698889 28040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:49:00.699029 28040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "d2475b56060e4d199613d36a6f51a867"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "944b5c5747aae188c91dc9cdbcc0ca01"
server_key_iv: "66f6bf83e7b0f91a768bccc642560684"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.699571 28040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:00.717054 28040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:00.720304 28040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:00.720562 28040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:00.727417 28040 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:34265
I20260809 07:49:00.727488 28113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:34265 every 8 connection(s)
I20260809 07:49:00.728549 28040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:00.732714 28114 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:49:00.734221 28114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Bootstrap starting.
I20260809 07:49:00.736661 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 28040
I20260809 07:49:00.737106 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:35851
--local_ip_for_outbound_sockets=127.18.71.65
--tserver_master_addrs=127.18.71.126:34265
--webserver_port=42187
--webserver_interface=127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:49:00.751386 28114 log.cc:824] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:00.757915 28114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:49:00.758304 28114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Bootstrap complete.
I20260809 07:49:00.762645 28114 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:49:00.763034 28114 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2475b56060e4d199613d36a6f51a867, State: Initialized, Role: FOLLOWER
I20260809 07:49:00.763533 28114 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:49:00.763646 28114 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:49:00.763767 28114 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:49:00.763862 28114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:49:00.765799 28114 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:49:00.766132 28114 leader_election.cc:302] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2475b56060e4d199613d36a6f51a867; no voters:
I20260809 07:49:00.766480 28114 leader_election.cc:288] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 07:49:00.766580 28118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:49:00.766796 28118 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [term 2 LEADER]: Becoming Leader. State: Replica: d2475b56060e4d199613d36a6f51a867, State: Running, Role: LEADER
I20260809 07:49:00.767077 28118 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } }
I20260809 07:49:00.767668 28114 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:00.767939 28118 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2475b56060e4d199613d36a6f51a867. Latest consensus state: current_term: 2 leader_uuid: "d2475b56060e4d199613d36a6f51a867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } } }
I20260809 07:49:00.768064 28118 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:00.768419 28122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:49:00.768519 28119 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d2475b56060e4d199613d36a6f51a867" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2475b56060e4d199613d36a6f51a867" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 34265 } } }
I20260809 07:49:00.768705 28119 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867 [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:00.771601 28122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:49:00.772219 28122 catalog_manager.cc:1259] Loaded cluster ID: 57c6eecc50a94eb099bad52872af48ed
I20260809 07:49:00.772300 28122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:49:00.774758 28122 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:49:00.775458 28122 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d2475b56060e4d199613d36a6f51a867: Loaded TSK: 0
I20260809 07:49:00.777594 28122 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 07:49:00.853587 28116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:00.853853 28116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:00.853928 28116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:00.857973 28116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:00.858062 28116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:00.858153 28116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:49:00.863315 28116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.65.keytab
--principal=oryx/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:35851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=42187
--tserver_master_addrs=127.18.71.126:34265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:00.864307 28116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:00.865357 28116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:00.876230 28144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:00.876233 28142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.876719 28116 server_base.cc:1034] running on GCE node
W20260809 07:49:00.876762 28146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:00.877122 28116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:00.877763 28116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:00.878963 28116 hybrid_clock.cc:648] HybridClock initialized: now 1786261740878938 us; error 48 us; skew 500 ppm
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261740, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:00.882117 28116 init.cc:377] Logged in from keytab as oryx/127.18.71.65@KRBTEST.COM (short username oryx)
I20260809 07:49:00.882987 28116 webserver.cc:460] Webserver started at http://127.18.71.65:42187/ using document root <none> and password file <none>
I20260809 07:49:00.883543 28116 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:00.883618 28116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:00.887658 28116 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:49:00.889858 28154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.891104 28116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:49:00.891247 28116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "9be0183a82a94520b3c6092f4ca551b2"
format_stamp: "Formatted at 2026-08-09 07:48:59 on dist-test-slave-10pc"
server_key: "e8b8e400d58bc34ab9f24c2d034c266f"
server_key_iv: "9bb5479a4ed2e34fcd9729773c717f4a"
server_key_version: "encryptionkey@0"
I20260809 07:49:00.891790 28116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:00.908679 28116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:00.912117 28116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:00.912328 28116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:00.912952 28116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:00.913796 28116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:00.913868 28116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.913942 28116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:00.914002 28116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:00.923022 28116 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:35851
I20260809 07:49:00.923094 28280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:35851 every 8 connection(s)
I20260809 07:49:00.924034 28116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:00.924628 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 28116
I20260809 07:49:00.925133 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:40937
--local_ip_for_outbound_sockets=127.18.71.66
--tserver_master_addrs=127.18.71.126:34265
--webserver_port=33857
--webserver_interface=127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:00.939270 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:00.926358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:50333)
0809 07:49:00.926947 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:00.927019 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:00.928200 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:00.929623 (+ 1423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:00.929635 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:00.930211 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:00.930958 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.930984 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.932325 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.932331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.934006 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.934021 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.934855 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.934864 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.935100 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.935767 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:00.935794 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:00.938531 (+ 2737us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":149,"threads_started":1}
W20260809 07:49:00.939776 28282 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:34265 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.71.126:34265: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
W20260809 07:49:00.940186 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:00.926326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:50333 (local address 127.18.71.126:34265)
0809 07:49:00.928656 (+ 2330us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:00.928671 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:00.928704 (+ 33us) server_negotiation.cc:407] Connection header received
0809 07:49:00.928908 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:00.928918 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:00.929203 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:00.929458 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:00.931164 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.932173 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.934167 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.934670 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.934803 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.938819 (+ 4016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:50333: BlockingRecv error: recv got EOF from 127.18.71.65:50333 (error 108)
Metrics: {"server-negotiator.queue_time_us":2188,"thread_start_us":135,"threads_started":1}
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:00.945575 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:00.940262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:33781)
0809 07:49:00.940646 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:00.940663 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:00.940747 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:00.941006 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:00.941008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:00.941136 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:00.941227 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.941231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.942218 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.942220 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.942734 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.942741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.943265 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.943267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.943332 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.943810 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:00.943821 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:00.945483 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":297}
W20260809 07:49:00.945757 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:00.940347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:33781 (local address 127.18.71.126:34265)
0809 07:49:00.940720 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:00.940725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:00.940740 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:49:00.940838 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:00.940841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:00.940899 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:00.940983 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:00.941340 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.942111 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.942833 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.943164 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.943255 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.945642 (+ 2387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:33781: BlockingRecv error: recv got EOF from 127.18.71.65:33781 (error 108)
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":155,"threads_started":1}
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:00 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:00.950937 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:00.946064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:45359)
0809 07:49:00.946149 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:00.946164 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:00.946240 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:00.946467 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:00.946487 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:00.946615 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:00.946676 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.946681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.947628 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.947630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.948131 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:00.948138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.948703 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.948706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:00.948762 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.949212 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:00.949223 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:00.950855 (+ 1632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 07:49:00.951097 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:00.946114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:45359 (local address 127.18.71.126:34265)
0809 07:49:00.946220 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:00.946224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:00.946237 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:49:00.946323 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:00.946325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:00.946367 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:00.946444 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:00.946781 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.947526 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.948238 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:00.948591 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:00.948700 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:00.950996 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:45359: BlockingRecv error: recv got EOF from 127.18.71.65:45359 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 07:49:00.951300 28282 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 07:49:01.043437 28287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:01.043679 28287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:01.043720 28287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:01.047744 28287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:01.047806 28287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:01.047894 28287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:49:01.053059 28287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.66.keytab
--principal=oryx/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:40937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=33857
--tserver_master_addrs=127.18.71.126:34265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:01.054136 28287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:01.055164 28287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:01.064752 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:01.064826 28287 server_base.cc:1034] running on GCE node
W20260809 07:49:01.064754 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:01.065029 28297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:01.065423 28287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:01.066045 28287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:01.067240 28287 hybrid_clock.cc:648] HybridClock initialized: now 1786261741067211 us; error 47 us; skew 500 ppm
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261741, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:01.070353 28287 init.cc:377] Logged in from keytab as oryx/127.18.71.66@KRBTEST.COM (short username oryx)
I20260809 07:49:01.071229 28287 webserver.cc:460] Webserver started at http://127.18.71.66:33857/ using document root <none> and password file <none>
I20260809 07:49:01.071784 28287 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:01.071894 28287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:01.075623 28287 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:49:01.077682 28306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.078795 28287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:49:01.078926 28287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "5fcfb5760d0e40c394c30f6837107238"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "064ac6fd2a1fd29a719c34db9ac6512e"
server_key_iv: "26eaa118ad18f76cf616279c5dfc1f81"
server_key_version: "encryptionkey@0"
I20260809 07:49:01.079470 28287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:01.096099 28287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:01.099017 28287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:01.099237 28287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:01.099841 28287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:01.100756 28287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:01.100826 28287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.100900 28287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:01.100952 28287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.110436 28287 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:40937
I20260809 07:49:01.110466 28434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:40937 every 8 connection(s)
I20260809 07:49:01.111420 28287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:01.113327 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 28287
I20260809 07:49:01.113830 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:37655
--local_ip_for_outbound_sockets=127.18.71.67
--tserver_master_addrs=127.18.71.126:34265
--webserver_port=45221
--webserver_interface=127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 07:49:01.123589 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:01.113430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:42667 (local address 127.18.71.126:34265)
0809 07:49:01.113567 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.113571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.114594 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 07:49:01.115941 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.115944 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.116008 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.116122 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.117405 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.118267 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.119554 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.119973 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.120110 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.123406 (+ 3296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:42667: BlockingRecv error: recv got EOF from 127.18.71.66:42667 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:49:01.124029 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.113870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:42667)
0809 07:49:01.114423 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:01.114471 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:01.115717 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:01.116305 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:01.116312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:01.116729 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:01.117221 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.117236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.118438 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.118441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.119409 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.119420 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.120137 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.120139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.120282 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.120954 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:01.120969 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:01.123207 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":166,"threads_started":1}
W20260809 07:49:01.124655 28435 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:34265 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.71.126:34265: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:01.130517 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.125140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:59095)
0809 07:49:01.125375 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:01.125386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:01.125446 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:01.125698 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:01.125700 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:01.125823 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:01.125906 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.125910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.126969 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.127006 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.127548 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.127556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.128147 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.128154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.128229 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.128751 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:01.128762 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:01.130431 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178}
W20260809 07:49:01.130695 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:01.125261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:59095 (local address 127.18.71.126:34265)
0809 07:49:01.125411 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.125417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.125435 (+ 18us) server_negotiation.cc:407] Connection header received
0809 07:49:01.125517 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.125523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.125578 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.125670 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.126015 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.126862 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.127662 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.128018 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.128117 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.130587 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:59095: BlockingRecv error: recv got EOF from 127.18.71.66:59095 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:01.136343 28437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.131216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:55963)
0809 07:49:01.131371 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:01.131383 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:01.131444 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:01.131651 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:01.131654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:01.131829 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:01.131909 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.131915 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.132616 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.132620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.133513 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.133527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.133966 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.133969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.134053 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.134475 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:01.134489 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:01.136216 (+ 1727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260809 07:49:01.136425 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:01.131310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:55963 (local address 127.18.71.126:34265)
0809 07:49:01.131411 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.131414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.131425 (+ 11us) server_negotiation.cc:407] Connection header received
0809 07:49:01.131523 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.131524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.131554 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.131623 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.132010 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.132525 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.133658 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.133866 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.133929 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.136317 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:55963: BlockingRecv error: recv got EOF from 127.18.71.66:55963 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 07:49:01.136672 28435 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 07:49:01.236716 28438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:01.237015 28438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:01.237099 28438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:01.241426 28438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:01.241533 28438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:01.241636 28438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:49:01.246853 28438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/krb5kdc/oryx_127.18.71.67.keytab
--principal=oryx/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:37655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=45221
--tserver_master_addrs=127.18.71.126:34265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:01.247951 28438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:01.249015 28438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:01.258762 28447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:01.258769 28443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:01.258822 28438 server_base.cc:1034] running on GCE node
W20260809 07:49:01.258769 28444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:01.259418 28438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:01.260038 28438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:01.261189 28438 hybrid_clock.cc:648] HybridClock initialized: now 1786261741261172 us; error 27 us; skew 500 ppm
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261741, etypes {rep=17 tkt=17 ses=17}, oryx/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:01.263981 28438 init.cc:377] Logged in from keytab as oryx/127.18.71.67@KRBTEST.COM (short username oryx)
I20260809 07:49:01.264858 28438 webserver.cc:460] Webserver started at http://127.18.71.67:45221/ using document root <none> and password file <none>
I20260809 07:49:01.265383 28438 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:01.265452 28438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:01.269327 28438 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:01.271420 28456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.272774 28438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:49:01.272894 28438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "2cd24f66284f42a8b2f947409527661f"
format_stamp: "Formatted at 2026-08-09 07:49:00 on dist-test-slave-10pc"
server_key: "19899b4c7918f980baafbc795886604a"
server_key_iv: "0887afea4b68c405422678aefed47675"
server_key_version: "encryptionkey@0"
I20260809 07:49:01.273418 28438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:01.295697 28438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:01.298904 28438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:01.299098 28438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:01.299676 28438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:01.300539 28438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:01.300587 28438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.300648 28438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:01.300686 28438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:01.309798 28438 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:37655
I20260809 07:49:01.309846 28580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:37655 every 8 connection(s)
I20260809 07:49:01.310814 28438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:01.312337 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 28438
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:01.322041 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.313377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.18.71.126:34265)
0809 07:49:01.313516 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.313530 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.313545 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:49:01.313598 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.313601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.313650 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.313747 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.314789 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.315330 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.316223 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.316502 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.316619 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.318059 (+ 1440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:01.318065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:01.318079 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:01.318103 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:01.319880 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:01.320344 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:01.320349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:01.320353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:01.320523 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:01.320832 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:01.320837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:01.320838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:01.321166 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:01.321366 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:01.321681 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 07:49:01.321923 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 07:49:01.322384 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:01.312683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:44325 (local address 127.18.71.126:34265)
0809 07:49:01.312847 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.312852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.313850 (+ 998us) server_negotiation.cc:407] Connection header received
0809 07:49:01.315421 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.315424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.315470 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.315547 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.316723 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.317379 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.318522 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.318887 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.318997 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.322287 (+ 3290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:44325: BlockingRecv error: recv got EOF from 127.18.71.67:44325 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 07:49:01.323935 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.313191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:44325)
0809 07:49:01.313692 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:01.313764 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:01.315221 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:01.315724 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:01.315731 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:01.316052 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:01.316584 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.316600 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.317502 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.317505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.318399 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.318413 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.319058 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.319065 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.319279 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.319794 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:01.319809 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:01.323009 (+ 3200us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":137,"threads_started":1}
W20260809 07:49:01.324483 28581 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:34265 (0 consecutive failures): Not authorized: Failed to ping master at 127.18.71.126:34265: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:01.329463 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.324915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:57143)
0809 07:49:01.325067 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:01.325082 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:01.325157 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:01.325373 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:01.325376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:01.325530 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:01.325615 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.325620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.326293 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.326298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.326830 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.326838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.327272 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.327276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.327368 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.327735 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:01.327746 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:01.329335 (+ 1589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 07:49:01.329563 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:01.324985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:57143 (local address 127.18.71.126:34265)
0809 07:49:01.325102 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.325108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.325123 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:49:01.325238 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.325241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.325281 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.325359 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.325723 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.326191 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.326950 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.327160 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.327268 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.329439 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:57143: BlockingRecv error: recv got EOF from 127.18.71.67:57143 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:01.334372 28585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.329939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:60799)
0809 07:49:01.330053 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:01.330065 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:01.330160 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:01.330384 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:01.330386 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:01.330527 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:01.330592 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.330596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.331284 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.331289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.331780 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.331787 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.332219 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.332223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.332307 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.332732 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:01.332743 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:01.334247 (+ 1504us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 07:49:01.334450 28285 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:01.330036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:60799 (local address 127.18.71.126:34265)
0809 07:49:01.330189 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.330194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.330220 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:49:01.330253 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.330256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.330293 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.330370 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.330724 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.331180 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.331901 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.332115 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.332216 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.334345 (+ 2129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:60799: BlockingRecv error: recv got EOF from 127.18.71.67:60799 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260809 07:49:01.334614 28581 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:01 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:01.957670 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:01.951986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:36945)
0809 07:49:01.952272 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:01.952294 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:01.952380 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:01.952616 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:01.952618 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:01.952804 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:01.952900 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.952923 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.953754 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.953757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.954435 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:01.954444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.955063 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.955070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:01.955164 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.955691 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:01.955708 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:01.957552 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
W20260809 07:49:01.957912 28587 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:01.952074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:36945 (local address 127.18.71.126:34265)
0809 07:49:01.952363 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:01.952367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:01.952380 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:49:01.952474 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:01.952476 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:01.952516 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:01.952581 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:01.953027 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.953653 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.954544 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:01.954921 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:01.955028 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:01.957749 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:36945: BlockingRecv error: recv got EOF from 127.18.71.65:36945 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":116,"threads_started":1}
Aug 09 07:49:02 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:02 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:02.143211 28588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:02.137365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:54129)
0809 07:49:02.137650 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:02.137664 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:02.137768 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:02.138014 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:02.138016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:02.138206 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:02.138308 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:02.138313 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.139125 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.139129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:02.139747 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:02.139756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.140195 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.140198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:02.140269 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:02.140917 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:02.140933 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:02.143088 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":94,"threads_started":1}
W20260809 07:49:02.143348 28587 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:02.137432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:54129 (local address 127.18.71.126:34265)
0809 07:49:02.137593 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:02.137597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:02.137736 (+ 139us) server_negotiation.cc:407] Connection header received
0809 07:49:02.137850 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:02.137852 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:02.137899 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:02.137973 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:02.138412 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.138937 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.139860 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.140073 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.140159 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:02.143229 (+ 3070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:54129: BlockingRecv error: recv got EOF from 127.18.71.66:54129 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 07:49:02 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:02 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:02.341605 28589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:02.335342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:58729)
0809 07:49:02.335746 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:02.335768 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:02.335918 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:02.336240 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:02.336243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:02.336506 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:02.336641 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:02.336648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.337446 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.337451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:02.338042 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:02.338051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.338547 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.338550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:02.338692 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:02.339322 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:02.339337 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:02.341488 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":92,"threads_started":1}
W20260809 07:49:02.341782 28587 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:02.335496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:58729 (local address 127.18.71.126:34265)
0809 07:49:02.335706 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:02.335711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:02.335854 (+ 143us) server_negotiation.cc:407] Connection header received
0809 07:49:02.336031 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:02.336034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:02.336115 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:02.336210 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:02.336785 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.337330 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.338166 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.338419 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.338530 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:02.341653 (+ 3123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:58729: BlockingRecv error: recv got EOF from 127.18.71.67:58729 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 07:49:02 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:02 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:02.963800 28590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:02.958519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:40369)
0809 07:49:02.958806 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:02.958820 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:02.958908 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:02.959162 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:02.959164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:02.959356 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:02.959459 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:02.959464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.960176 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.960178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:02.960808 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:02.960817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.961242 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.961245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:02.961312 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:02.961813 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:02.961824 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:02.963669 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
W20260809 07:49:02.963987 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:02.958585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:40369 (local address 127.18.71.126:34265)
0809 07:49:02.958841 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:02.958845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:02.958860 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:49:02.958991 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:02.958993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:02.959037 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:02.959122 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:02.959556 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.960083 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.960910 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:02.961138 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:02.961215 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:02.963835 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:40369: BlockingRecv error: recv got EOF from 127.18.71.65:40369 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
Aug 09 07:49:03 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:03 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:03.150035 28592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:03.144239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:38647)
0809 07:49:03.144569 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:03.144582 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:03.144716 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:03.145007 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:03.145010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:03.145249 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:03.145351 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:03.145357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.146140 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.146145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:03.146703 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:03.146711 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.147176 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.147179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:03.147267 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:03.147868 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:03.147880 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:03.149893 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":93,"threads_started":1}
W20260809 07:49:03.150231 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:03.144291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:38647 (local address 127.18.71.126:34265)
0809 07:49:03.144486 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:03.144490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:03.144660 (+ 170us) server_negotiation.cc:407] Connection header received
0809 07:49:03.144821 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:03.144824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:03.144879 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:03.144983 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:03.145477 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.146020 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.146827 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.147058 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.147170 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:03.150080 (+ 2910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:38647: BlockingRecv error: recv got EOF from 127.18.71.66:38647 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 07:49:03 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:03 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:03.352222 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:03.342436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:48785)
0809 07:49:03.342773 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:03.342790 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:03.342908 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:03.343223 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:03.343226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:03.343445 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:03.343536 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:03.343541 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.344321 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.344326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:03.345005 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:03.345014 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.345473 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.345476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:03.345560 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:03.350027 (+ 4467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:03.350074 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:03.352047 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":120,"threads_started":1}
W20260809 07:49:03.352294 28591 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:03.342612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:48785 (local address 127.18.71.126:34265)
0809 07:49:03.342773 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:03.342779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:03.342851 (+ 72us) server_negotiation.cc:407] Connection header received
0809 07:49:03.343021 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:03.343024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:03.343079 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:03.343173 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:03.343659 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.344203 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.345136 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.345357 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.345468 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:03.352185 (+ 6717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:48785: BlockingRecv error: recv got EOF from 127.18.71.67:48785 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 07:49:03 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:03 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:03.970156 28594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:03.964646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:45237)
0809 07:49:03.964932 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:03.964965 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:03.965073 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:03.965327 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:03.965329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:03.965514 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:03.965613 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:03.965618 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.966329 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.966332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:03.966877 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:03.966884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.967303 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.967306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:03.967385 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:03.967920 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:03.967931 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:03.969987 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":79,"threads_started":1}
W20260809 07:49:03.970398 28595 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:03.964779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:45237 (local address 127.18.71.126:34265)
0809 07:49:03.965038 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:03.965042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:03.965055 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:49:03.965165 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:03.965168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:03.965211 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:03.965285 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:03.965708 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.966238 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.966972 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:03.967194 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:03.967277 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:03.970123 (+ 2846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:45237: BlockingRecv error: recv got EOF from 127.18.71.65:45237 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":80,"threads_started":1}
Aug 09 07:49:04 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:04 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:04.157251 28596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:04.150872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:36417)
0809 07:49:04.151204 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:04.151228 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:04.151382 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:04.151688 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:04.151691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:04.151923 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:04.152029 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:04.152035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.152964 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.152969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:04.153592 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:04.153601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.154130 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.154134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:04.154238 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:04.154732 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:04.154744 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:04.157046 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":98,"threads_started":1}
W20260809 07:49:04.157450 28595 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:04.150928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:36417 (local address 127.18.71.126:34265)
0809 07:49:04.151174 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:04.151179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:04.151313 (+ 134us) server_negotiation.cc:407] Connection header received
0809 07:49:04.151506 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:04.151510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:04.151570 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:04.151666 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:04.152156 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.152839 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.153725 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.153998 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.154122 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:04.157327 (+ 3205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:36417: BlockingRecv error: recv got EOF from 127.18.71.66:36417 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 09 07:49:04 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:04 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:04.358795 28597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:04.353106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:38403)
0809 07:49:04.353408 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:04.353421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:04.353557 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:04.353846 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:04.353849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:04.354061 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:04.354167 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:04.354172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.354935 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.354940 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:04.355521 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:04.355529 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.355990 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.355994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:04.356081 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:04.356694 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:04.356706 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:04.358615 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
W20260809 07:49:04.358870 28595 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:04.353176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:38403 (local address 127.18.71.126:34265)
0809 07:49:04.353354 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:04.353360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:04.353499 (+ 139us) server_negotiation.cc:407] Connection header received
0809 07:49:04.353670 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:04.353673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:04.353730 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:04.353819 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:04.354291 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.354822 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.355650 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.355873 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.355985 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:04.358752 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:38403: BlockingRecv error: recv got EOF from 127.18.71.67:38403 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 07:49:04 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:04 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:04.976807 28598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:04.970992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:40063)
0809 07:49:04.971277 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:04.971291 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:04.971381 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:04.971648 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:04.971651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:04.971832 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:04.971931 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:04.971936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.972754 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.972756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:04.973429 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:04.973440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.974044 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.974049 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:04.974155 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:04.974720 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:04.974731 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:04.976664 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":117,"threads_started":1}
W20260809 07:49:04.977007 28599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:04.971102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:40063 (local address 127.18.71.126:34265)
0809 07:49:04.971358 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:04.971362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:04.971377 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:49:04.971468 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:04.971470 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:04.971515 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:04.971600 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:04.972057 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.972655 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.973618 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:04.973920 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:04.974026 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:04.976864 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:40063: BlockingRecv error: recv got EOF from 127.18.71.65:40063 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":107,"threads_started":1}
Aug 09 07:49:05 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:05 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:05.168020 28600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:05.158126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:33753)
0809 07:49:05.158455 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:05.158472 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:05.158591 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:05.158878 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:05.158882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:05.159098 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:05.159248 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:05.159256 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.160350 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.160353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:05.160929 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:05.160937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.161370 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.161372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:05.161436 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:05.165803 (+ 4367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:05.165815 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:05.167817 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":95,"threads_started":1}
W20260809 07:49:05.168164 28599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:05.158270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:33753 (local address 127.18.71.126:34265)
0809 07:49:05.158435 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:05.158440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:05.158533 (+ 93us) server_negotiation.cc:407] Connection header received
0809 07:49:05.158696 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:05.158699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:05.158753 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:05.158851 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:05.159404 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.160231 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.161048 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.161264 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.161346 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:05.167989 (+ 6643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:33753: BlockingRecv error: recv got EOF from 127.18.71.66:33753 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 07:49:05 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:05 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:05.366114 28601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:05.359536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:42943)
0809 07:49:05.359896 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:05.359908 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:05.360033 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:05.360328 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:05.360331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:05.360582 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:05.360686 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:05.360692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.361567 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.361571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:05.362343 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:05.362353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.363067 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.363070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:05.363154 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:05.363719 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:05.363733 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:05.365999 (+ 2266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":77,"threads_started":1}
W20260809 07:49:05.366291 28599 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:05.359615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:42943 (local address 127.18.71.126:34265)
0809 07:49:05.359772 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:05.359777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:05.359977 (+ 200us) server_negotiation.cc:407] Connection header received
0809 07:49:05.360144 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:05.360148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:05.360222 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:05.360303 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:05.360801 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.361471 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.362477 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.362835 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.362905 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:05.366177 (+ 3272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:42943: BlockingRecv error: recv got EOF from 127.18.71.67:42943 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 07:49:05 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:05 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:05.983517 28602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:05.977730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:58785)
0809 07:49:05.978134 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:05.978148 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:05.978251 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:05.978532 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:05.978535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:05.978775 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:05.978880 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:05.978886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.979638 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.979640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:05.980209 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:05.980216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.980704 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.980707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:05.980777 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:05.981299 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:05.981312 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:05.983385 (+ 2073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":113,"threads_started":1}
W20260809 07:49:05.983836 28604 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:05.977864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:58785 (local address 127.18.71.126:34265)
0809 07:49:05.978134 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:05.978137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:05.978225 (+ 88us) server_negotiation.cc:407] Connection header received
0809 07:49:05.978342 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:05.978344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:05.978389 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:05.978469 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:05.978992 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.979531 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.980333 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:05.980595 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:05.980696 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:05.983683 (+ 2987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:58785: BlockingRecv error: recv got EOF from 127.18.71.65:58785 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":66,"threads_started":1}
Aug 09 07:49:06 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:06 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:06.174208 28605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:06.169011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:35215)
0809 07:49:06.169315 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:06.169329 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:06.169429 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:06.169695 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:06.169697 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:06.169890 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:06.169989 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:06.169994 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.170718 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.170721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:06.171266 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:06.171273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.171706 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.171709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:06.171771 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:06.172269 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:06.172281 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:06.174095 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
W20260809 07:49:06.174394 28604 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:06.169179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:35215 (local address 127.18.71.126:34265)
0809 07:49:06.169344 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:06.169348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:06.169401 (+ 53us) server_negotiation.cc:407] Connection header received
0809 07:49:06.169528 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:06.169531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:06.169577 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:06.169677 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:06.170091 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.170610 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.171375 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.171592 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.171685 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:06.174227 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:35215: BlockingRecv error: recv got EOF from 127.18.71.66:35215 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 07:49:06 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:06 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:06.372855 28607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:06.367048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:48389)
0809 07:49:06.367499 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:06.367513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:06.367683 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:06.367921 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:06.367923 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:06.368158 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:06.368253 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:06.368259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.369200 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.369203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:06.369798 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:06.369808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.370248 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.370251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:06.370314 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:06.370789 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:06.370801 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:06.372741 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":102,"threads_started":1}
W20260809 07:49:06.373018 28604 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:06.367262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:48389 (local address 127.18.71.126:34265)
0809 07:49:06.367445 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:06.367450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:06.367627 (+ 177us) server_negotiation.cc:407] Connection header received
0809 07:49:06.367768 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:06.367770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:06.367816 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:06.367884 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:06.368364 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.369096 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.369919 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.370143 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.370225 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:06.372889 (+ 2664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:48389: BlockingRecv error: recv got EOF from 127.18.71.67:48389 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 07:49:06 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:06 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:06.990167 28608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:06.984380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:51791)
0809 07:49:06.984722 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:06.984744 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:06.984868 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:06.985173 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:06.985176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:06.985398 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:06.985497 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:06.985503 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.986282 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.986287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:06.986873 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:06.986881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.987361 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.987364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:06.987456 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:06.988048 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:06.988059 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:06.990050 (+ 1991us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
W20260809 07:49:06.990361 28609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:06.984429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:51791 (local address 127.18.71.126:34265)
0809 07:49:06.984734 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:06.984739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:06.984803 (+ 64us) server_negotiation.cc:407] Connection header received
0809 07:49:06.984981 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:06.984984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:06.985030 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:06.985134 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:06.985642 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.986170 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.987007 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:06.987244 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:06.987365 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:06.990186 (+ 2821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:51791: BlockingRecv error: recv got EOF from 127.18.71.65:51791 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":63,"threads_started":1}
Aug 09 07:49:07 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:07 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:07.180814 28610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:07.174984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:43261)
0809 07:49:07.175283 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:07.175295 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:07.175427 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:07.175728 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:07.175731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:07.175952 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:07.176071 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:07.176077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.176930 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.176934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:07.177572 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:07.177582 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.178056 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.178059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:07.178148 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:07.178661 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:07.178673 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:07.180665 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":106,"threads_started":1}
W20260809 07:49:07.180917 28609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:07.175048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:43261 (local address 127.18.71.126:34265)
0809 07:49:07.175215 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:07.175221 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:07.175366 (+ 145us) server_negotiation.cc:407] Connection header received
0809 07:49:07.175543 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:07.175547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:07.175601 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:07.175684 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:07.176190 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.176820 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.177712 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.177939 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.178049 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:07.180780 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:43261: BlockingRecv error: recv got EOF from 127.18.71.66:43261 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 07:49:07 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:07 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:07.379750 28611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:07.373838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:47835)
0809 07:49:07.374137 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:07.374151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:07.374287 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:07.374562 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:07.374565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:07.374784 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:07.374878 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:07.374884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.375708 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.375713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:07.376272 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:07.376281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.376843 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.376847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:07.376937 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:07.377651 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:07.377664 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:07.379634 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
W20260809 07:49:07.379890 28609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:07.373902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:47835 (local address 127.18.71.126:34265)
0809 07:49:07.374079 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:07.374084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:07.374223 (+ 139us) server_negotiation.cc:407] Connection header received
0809 07:49:07.374385 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:07.374387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:07.374442 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:07.374532 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:07.374994 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.375589 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.376413 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.376716 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.376836 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:07.379769 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:47835: BlockingRecv error: recv got EOF from 127.18.71.67:47835 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 07:49:07 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:07 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:07.996668 28613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:07.990933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:41717)
0809 07:49:07.991297 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:07.991311 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:07.991411 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:07.991667 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:07.991669 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:07.991861 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:07.991963 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:07.991968 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.992767 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.992770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:07.993323 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:07.993330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.993764 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.993767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:07.993840 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:07.994480 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:07.994492 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:07.996550 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":100,"threads_started":1}
W20260809 07:49:07.996836 28615 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:07.991044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:41717 (local address 127.18.71.126:34265)
0809 07:49:07.991304 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:07.991308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:07.991385 (+ 77us) server_negotiation.cc:407] Connection header received
0809 07:49:07.991493 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:07.991495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:07.991543 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:07.991624 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:07.992093 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.992664 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.993431 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:07.993663 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:07.993744 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:07.996664 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:41717: BlockingRecv error: recv got EOF from 127.18.71.65:41717 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":67,"threads_started":1}
Aug 09 07:49:08 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:08 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:08.187263 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:08.181575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:49451)
0809 07:49:08.181947 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:08.181960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:08.182093 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:08.182373 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:08.182375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:08.182607 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:08.182719 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:08.182726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.183508 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.183513 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:08.184079 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:08.184088 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.184608 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.184619 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:08.184714 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:08.185225 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:08.185237 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:08.187147 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":111,"threads_started":1}
W20260809 07:49:08.187647 28615 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:08.181647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:49451 (local address 127.18.71.126:34265)
0809 07:49:08.181820 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:08.181825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:08.182059 (+ 234us) server_negotiation.cc:407] Connection header received
0809 07:49:08.182218 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:08.182220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:08.182264 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:08.182348 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:08.182838 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.183396 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.184209 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.184438 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.184606 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:08.187525 (+ 2919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:49451: BlockingRecv error: recv got EOF from 127.18.71.66:49451 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 07:49:08 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:08 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:08.391247 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:08.380598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:50395)
0809 07:49:08.380969 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:08.380983 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:08.381094 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:08.381352 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:08.381354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:08.381540 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:08.381654 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:08.381659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.382365 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.382367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:08.382991 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:08.383001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.383473 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.383476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:08.383541 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:08.388857 (+ 5316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:08.388872 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:08.391110 (+ 2238us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":107,"threads_started":1}
W20260809 07:49:08.391515 28615 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:08.380686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:50395 (local address 127.18.71.126:34265)
0809 07:49:08.380885 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:08.380891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:08.381075 (+ 184us) server_negotiation.cc:407] Connection header received
0809 07:49:08.381183 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:08.381185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:08.381238 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:08.381314 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:08.381756 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.382270 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.383128 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.383353 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.383443 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:08.391335 (+ 7892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:50395: BlockingRecv error: recv got EOF from 127.18.71.67:50395 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 07:49:08 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:08 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:09.003198 28619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:08.997422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:42953)
0809 07:49:08.997742 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:08.997759 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:08.997864 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:08.998103 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:08.998105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:08.998289 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:08.998387 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:08.998391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.999084 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.999086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:08.999625 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:08.999632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.000254 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.000258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:09.000369 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:09.000931 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:09.000943 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:09.003082 (+ 2139us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":71,"threads_started":1}
W20260809 07:49:09.003434 28620 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:08.997614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:42953 (local address 127.18.71.126:34265)
0809 07:49:08.997858 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:08.997862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:08.997876 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:49:08.997946 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:08.997948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:08.997987 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:08.998061 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:08.998492 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:08.998995 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:08.999742 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.000097 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.000224 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:09.003233 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:42953: BlockingRecv error: recv got EOF from 127.18.71.65:42953 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":81,"threads_started":1}
Aug 09 07:49:09 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:09 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:09.194416 28621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:09.188253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:36245)
0809 07:49:09.188637 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:09.188653 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:09.188778 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:09.189074 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:09.189077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:09.189307 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:09.189452 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:09.189460 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.190391 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.190396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:09.190974 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:09.190981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.191460 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.191464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:09.191551 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:09.192184 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:09.192195 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:09.194251 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":125,"threads_started":1}
W20260809 07:49:09.194521 28620 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:09.188303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:36245 (local address 127.18.71.126:34265)
0809 07:49:09.188526 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:09.188531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:09.188735 (+ 204us) server_negotiation.cc:407] Connection header received
0809 07:49:09.188894 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:09.188898 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:09.188956 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:09.189048 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:09.189583 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.190274 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.191098 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.191334 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.191457 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:09.194407 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:36245: BlockingRecv error: recv got EOF from 127.18.71.66:36245 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 07:49:09 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:09 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:09.397673 28623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:09.392147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:33139)
0809 07:49:09.392508 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:09.392526 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:09.392668 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:09.392960 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:09.392962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:09.393155 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:09.393252 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:09.393257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.393974 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.393976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:09.394538 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:09.394545 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.395019 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.395022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:09.395084 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:09.395559 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:09.395571 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:09.397518 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":130,"threads_started":1}
W20260809 07:49:09.397899 28620 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:09.392215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:33139 (local address 127.18.71.126:34265)
0809 07:49:09.392368 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:09.392372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:09.392596 (+ 224us) server_negotiation.cc:407] Connection header received
0809 07:49:09.392784 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:09.392787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:09.392852 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:09.392921 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:09.393353 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.393883 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.394639 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:09.394918 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:09.395003 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:09.397665 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:33139: BlockingRecv error: recv got EOF from 127.18.71.67:33139 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 07:49:10 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:10 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:10.009573 28624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:10.003971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:49421)
0809 07:49:10.004262 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:10.004276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:10.004372 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:10.004653 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:10.004656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:10.004844 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:10.004968 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:10.004974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.005715 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.005718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:10.006276 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:10.006284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.006792 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.006796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:10.006873 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:10.007402 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:10.007414 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:10.009408 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":101,"threads_started":1}
W20260809 07:49:10.009647 28625 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:10.004051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:49421 (local address 127.18.71.126:34265)
0809 07:49:10.004324 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:10.004328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:10.004342 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:49:10.004478 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:10.004480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:10.004527 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:10.004604 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:10.005078 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.005617 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.006381 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.006614 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.006695 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:10.009542 (+ 2847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:49421: BlockingRecv error: recv got EOF from 127.18.71.65:49421 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
Aug 09 07:49:10 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:10 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:10.201442 28626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:10.195486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:37793)
0809 07:49:10.195784 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:10.195797 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:10.195898 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:10.196195 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:10.196197 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:10.196436 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:10.196607 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:10.196614 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.197378 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.197381 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:10.197969 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:10.197978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.198440 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.198444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:10.198525 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:10.199063 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:10.199075 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:10.201263 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":112,"threads_started":1}
W20260809 07:49:10.201769 28625 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:10.195551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:37793 (local address 127.18.71.126:34265)
0809 07:49:10.195703 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:10.195708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:10.195878 (+ 170us) server_negotiation.cc:407] Connection header received
0809 07:49:10.195984 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:10.195988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:10.196046 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:10.196140 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:10.196730 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.197260 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.198088 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.198309 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.198384 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:10.201524 (+ 3140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:37793: BlockingRecv error: recv got EOF from 127.18.71.66:37793 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 07:49:10 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:10 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:10.404198 28627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:10.398470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:38311)
0809 07:49:10.398791 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:10.398807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:10.398918 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:10.399206 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:10.399208 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:10.399386 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:10.399481 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:10.399486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.400237 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.400240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:10.400818 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:10.400826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.401270 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.401272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:10.401337 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:10.401898 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:10.401911 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:10.404054 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":83,"threads_started":1}
W20260809 07:49:10.404261 28625 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:10.398563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:38311 (local address 127.18.71.126:34265)
0809 07:49:10.398690 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:10.398695 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:10.398971 (+ 276us) server_negotiation.cc:407] Connection header received
0809 07:49:10.399059 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:10.399062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:10.399111 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:10.399192 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:10.399588 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.400138 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.400916 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:10.401169 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:10.401249 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:10.404169 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:38311: BlockingRecv error: recv got EOF from 127.18.71.67:38311 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 07:49:11 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:11 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:11.016260 28628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:11.010650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:39793)
0809 07:49:11.011061 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:11.011079 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:11.011199 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:11.011515 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:11.011520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:11.011794 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:11.011931 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:11.011937 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.012708 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.012711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:11.013280 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:11.013288 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.013731 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.013733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:11.013799 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:11.014292 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:11.014302 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:11.016145 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":90,"threads_started":1}
W20260809 07:49:11.016508 28629 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:11.010728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:39793 (local address 127.18.71.126:34265)
0809 07:49:11.011032 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:11.011036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:11.011161 (+ 125us) server_negotiation.cc:407] Connection header received
0809 07:49:11.011311 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:11.011314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:11.011378 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:11.011467 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:11.012040 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.012608 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.013400 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.013631 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.013712 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:11.016324 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:39793: BlockingRecv error: recv got EOF from 127.18.71.65:39793 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
Aug 09 07:49:11 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:11 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:11.208609 28630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:11.202305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:59253)
0809 07:49:11.202592 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:11.202615 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:11.202713 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:11.202932 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:11.202934 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:11.203127 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:11.203285 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:11.203293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.204066 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.204069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:11.204861 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:11.204874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.205368 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.205371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:11.205441 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:11.206094 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:11.206107 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:11.208420 (+ 2313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":120,"threads_started":1}
W20260809 07:49:11.208828 28629 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:11.202387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:59253 (local address 127.18.71.126:34265)
0809 07:49:11.202550 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:11.202554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:11.202691 (+ 137us) server_negotiation.cc:407] Connection header received
0809 07:49:11.202781 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:11.202799 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:11.202844 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:11.202914 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:11.203397 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.203974 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.205038 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.205262 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.205338 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:11.208700 (+ 3362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:59253: BlockingRecv error: recv got EOF from 127.18.71.66:59253 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 07:49:11 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:11 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:11.411155 28631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:11.405358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:52859)
0809 07:49:11.405789 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:11.405803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:11.405905 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:11.406136 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:11.406138 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:11.406322 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:11.406417 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:11.406422 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.407273 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.407276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:11.407926 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:11.407936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.408443 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.408446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:11.408570 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:11.409063 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:11.409076 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:11.411000 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":91,"threads_started":1}
W20260809 07:49:11.411357 28629 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:11.405647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:52859 (local address 127.18.71.126:34265)
0809 07:49:11.405793 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:11.405797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:11.405870 (+ 73us) server_negotiation.cc:407] Connection header received
0809 07:49:11.405975 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:11.405977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:11.406024 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:11.406097 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:11.406550 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.407144 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.408074 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:11.408340 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:11.408424 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:11.411215 (+ 2791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:52859: BlockingRecv error: recv got EOF from 127.18.71.67:52859 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 07:49:12 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:12 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:12.022864 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:12.017253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:37625)
0809 07:49:12.017564 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:12.017578 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:12.017682 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:12.017967 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:12.017970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:12.018173 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:12.018293 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:12.018298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.019105 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.019108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:12.019673 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:12.019681 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.020107 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.020110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:12.020175 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:12.020734 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:12.020746 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:12.022748 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
W20260809 07:49:12.023029 28634 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:12.017326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:37625 (local address 127.18.71.126:34265)
0809 07:49:12.017586 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:12.017590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:12.017632 (+ 42us) server_negotiation.cc:407] Connection header received
0809 07:49:12.017771 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:12.017773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:12.017818 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:12.017903 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:12.018410 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.018978 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.019781 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.020002 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.020078 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:12.022892 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:37625: BlockingRecv error: recv got EOF from 127.18.71.65:37625 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":102,"threads_started":1}
Aug 09 07:49:12 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:12 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:12.217962 28635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:12.210642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:45097)
0809 07:49:12.210999 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:12.211014 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:12.211125 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:12.211347 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:12.211349 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:12.211549 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:12.211662 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:12.211668 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.212722 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.212729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:12.213537 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:12.213549 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.214136 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.214140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:12.214237 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:12.215714 (+ 1477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:12.215735 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:12.217837 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":136,"threads_started":1}
W20260809 07:49:12.218066 28634 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:12.210685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:45097 (local address 127.18.71.126:34265)
0809 07:49:12.210825 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:12.210830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:12.211083 (+ 253us) server_negotiation.cc:407] Connection header received
0809 07:49:12.211193 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:12.211195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:12.211238 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:12.211304 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:12.211764 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.212606 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.213653 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.213991 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.214102 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:12.217974 (+ 3872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:45097: BlockingRecv error: recv got EOF from 127.18.71.66:45097 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 07:49:12 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:12 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:12.417436 28636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:12.411969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:38495)
0809 07:49:12.412264 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:12.412277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:12.412374 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:12.412613 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:12.412615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:12.412798 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:12.412897 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:12.412902 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.413622 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.413625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:12.414190 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:12.414197 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.414633 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.414636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:12.414710 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:12.415214 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:12.415226 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:12.417319 (+ 2093us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":112,"threads_started":1}
W20260809 07:49:12.417600 28634 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:12.412053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:38495 (local address 127.18.71.126:34265)
0809 07:49:12.412202 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:12.412206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:12.412328 (+ 122us) server_negotiation.cc:407] Connection header received
0809 07:49:12.412439 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:12.412441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:12.412507 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:12.412574 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:12.412997 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.413517 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.414289 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:12.414489 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:12.414582 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:12.417448 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:38495: BlockingRecv error: recv got EOF from 127.18.71.67:38495 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 07:49:13 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:13 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:13.029798 28637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:13.023908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:33357)
0809 07:49:13.024223 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:13.024237 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:13.024329 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:13.024605 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:13.024607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:13.024803 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:13.024904 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:13.024909 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.025808 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.025810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:13.026377 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:13.026386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.026874 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.026876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:13.026942 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:13.027522 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:13.027535 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:13.029679 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":112,"threads_started":1}
W20260809 07:49:13.029959 28638 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:13.023998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:33357 (local address 127.18.71.126:34265)
0809 07:49:13.024313 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:13.024317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:13.024334 (+ 17us) server_negotiation.cc:407] Connection header received
0809 07:49:13.024399 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:13.024401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:13.024468 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:13.024568 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:13.025069 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.025682 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.026510 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.026763 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.026854 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:13.029824 (+ 2970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:33357: BlockingRecv error: recv got EOF from 127.18.71.65:33357 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":144,"threads_started":1}
Aug 09 07:49:13 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:13 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:13.225106 28639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:13.218871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:32933)
0809 07:49:13.219299 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:13.219315 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:13.219449 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:13.219900 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:13.219905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:13.220198 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:13.220332 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:13.220341 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.221361 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.221364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:13.221923 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:13.221931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.222390 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.222393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:13.222459 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:13.222979 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:13.222991 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:13.224933 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":85,"threads_started":1}
W20260809 07:49:13.225270 28638 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:13.219038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:32933 (local address 127.18.71.126:34265)
0809 07:49:13.219228 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:13.219234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:13.219418 (+ 184us) server_negotiation.cc:407] Connection header received
0809 07:49:13.219650 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:13.219653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:13.219756 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:13.219875 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:13.220523 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.221252 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.222054 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.222278 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.222375 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:13.225130 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:32933: BlockingRecv error: recv got EOF from 127.18.71.66:32933 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 07:49:13 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:13 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:13.427932 28640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:13.418226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:46603)
0809 07:49:13.418521 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:13.418537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:13.418646 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:13.418890 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:13.418892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:13.419105 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:13.419202 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:13.419207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.419938 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.419940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:13.420620 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:13.420629 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.421084 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.421086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:13.421147 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:13.425709 (+ 4562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:13.425722 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:13.427764 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":115,"threads_started":1}
W20260809 07:49:13.428112 28638 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:13.418296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:46603 (local address 127.18.71.126:34265)
0809 07:49:13.418456 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:13.418460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:13.418623 (+ 163us) server_negotiation.cc:407] Connection header received
0809 07:49:13.418729 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:13.418732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:13.418781 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:13.418852 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:13.419306 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.419839 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.420743 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:13.420982 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:13.421065 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:13.427917 (+ 6852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:46603: BlockingRecv error: recv got EOF from 127.18.71.67:46603 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 07:49:14 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:14 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:14.037154 28641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:14.030778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:38621)
0809 07:49:14.031130 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:14.031148 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:14.031259 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:14.031790 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:14.031793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:14.032031 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:14.032128 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:14.032134 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.033026 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.033030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:14.033614 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:14.033628 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.034147 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.034151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:14.034241 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:14.034804 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:14.034816 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:14.037024 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":141,"threads_started":1}
W20260809 07:49:14.037421 28642 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:14.030884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:38621 (local address 127.18.71.126:34265)
0809 07:49:14.031254 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:14.031261 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:14.031278 (+ 17us) server_negotiation.cc:407] Connection header received
0809 07:49:14.031507 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:14.031510 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:14.031573 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:14.031676 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:14.032249 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.032906 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.033770 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.034000 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.034142 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:14.037181 (+ 3039us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:38621: BlockingRecv error: recv got EOF from 127.18.71.65:38621 (error 108)
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":91,"threads_started":1}
Aug 09 07:49:14 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:14 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:14.232131 28643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:14.225977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:41307)
0809 07:49:14.226291 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:14.226305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:14.226407 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:14.226633 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:14.226635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:14.226819 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:14.226918 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:14.226924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.227776 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.227779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:14.228369 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:14.228377 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.229037 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.229040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:14.229148 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:14.229856 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:14.229871 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:14.231935 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":91,"threads_started":1}
W20260809 07:49:14.232177 28642 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:14.226096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:41307 (local address 127.18.71.126:34265)
0809 07:49:14.226300 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:14.226304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:14.226356 (+ 52us) server_negotiation.cc:407] Connection header received
0809 07:49:14.226482 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:14.226484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:14.226528 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:14.226599 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:14.227051 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.227671 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.228572 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.228858 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.228983 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:14.232064 (+ 3081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:41307: BlockingRecv error: recv got EOF from 127.18.71.66:41307 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 07:49:14 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:14 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:14.434458 28644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:14.428737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:41575)
0809 07:49:14.429100 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:14.429114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:14.429256 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:14.429572 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:14.429575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:14.429804 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:14.429898 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:14.429903 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.430687 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.430693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:14.431364 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:14.431373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.431852 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.431856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:14.431945 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:14.432481 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:14.432495 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:14.434339 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":91,"threads_started":1}
W20260809 07:49:14.434652 28642 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:14.428871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:41575 (local address 127.18.71.126:34265)
0809 07:49:14.429046 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:14.429051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:14.429195 (+ 144us) server_negotiation.cc:407] Connection header received
0809 07:49:14.429373 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:14.429377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:14.429432 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:14.429548 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:14.430026 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.430565 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.431503 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:14.431733 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:14.431845 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:14.434480 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:41575: BlockingRecv error: recv got EOF from 127.18.71.67:41575 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 07:49:15 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:15 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:15.043835 28645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:15.038055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:47269)
0809 07:49:15.038519 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:15.038533 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:15.038662 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:15.039053 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:15.039056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:15.039253 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:15.039352 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:15.039358 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.040087 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.040090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:15.040740 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:15.040749 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.041233 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.041235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:15.041299 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:15.041809 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:15.041820 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:15.043707 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":96,"threads_started":1}
W20260809 07:49:15.043999 28646 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:15.038098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:47269 (local address 127.18.71.126:34265)
0809 07:49:15.038480 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:15.038485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:15.038607 (+ 122us) server_negotiation.cc:407] Connection header received
0809 07:49:15.038775 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:15.038778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:15.038859 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:15.038947 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:15.039462 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.039978 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.040855 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.041126 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.041208 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:15.043837 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:47269: BlockingRecv error: recv got EOF from 127.18.71.65:47269 (error 108)
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":76,"threads_started":1}
Aug 09 07:49:15 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:15 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 07:49:15.239162 28646 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:15.233151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:35851 (local address 127.18.71.126:34265)
0809 07:49:15.233315 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:15.233320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:15.233445 (+ 125us) server_negotiation.cc:407] Connection header received
0809 07:49:15.233615 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:15.233618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:15.233672 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:15.233753 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:15.234242 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.234920 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.235999 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.236224 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.236305 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:15.239058 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:35851: BlockingRecv error: recv got EOF from 127.18.71.66:35851 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 07:49:15.239171 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:15.233054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:35851)
0809 07:49:15.233351 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:15.233367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:15.233501 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:15.233780 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:15.233783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:15.234022 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:15.234122 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:15.234129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.235053 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.235057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:15.235887 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:15.235898 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.236328 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.236330 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:15.236394 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:15.236929 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:15.236943 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:15.238930 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":102,"threads_started":1}
Aug 09 07:49:15 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:15 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.67@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:15.440944 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:15.435173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.67:44889)
0809 07:49:15.435491 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:15.435507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:15.435652 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:15.435945 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:15.435948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:15.436197 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:15.436293 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:15.436299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.437139 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.437143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:15.437720 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:15.437728 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.438198 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.438202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:15.438287 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:15.438781 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:15.438793 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:15.440833 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":87,"threads_started":1}
W20260809 07:49:15.441150 28646 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:15.435265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:44889 (local address 127.18.71.126:34265)
0809 07:49:15.435437 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:15.435442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:15.435591 (+ 149us) server_negotiation.cc:407] Connection header received
0809 07:49:15.435768 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:15.435777 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:15.435843 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:15.435921 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:15.436416 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.437022 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.437850 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:15.438084 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:15.438191 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:15.440997 (+ 2806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.67:44889: BlockingRecv error: recv got EOF from 127.18.71.67:44889 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 07:49:16 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:16 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.65@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:16.050645 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:16.044818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.65:40829)
0809 07:49:16.045181 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:16.045198 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:16.045311 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:16.045611 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:16.045614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:16.045836 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:16.045962 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:16.045972 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.046746 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.046749 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:16.047372 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:16.047381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.047876 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.047880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:16.047972 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:16.048553 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:16.048564 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:16.050520 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":97,"threads_started":1}
W20260809 07:49:16.050823 28651 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:16.044958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:40829 (local address 127.18.71.126:34265)
0809 07:49:16.045207 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:16.045212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:16.045255 (+ 43us) server_negotiation.cc:407] Connection header received
0809 07:49:16.045412 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:16.045416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:16.045474 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:16.045581 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:16.046070 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.046619 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.047505 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.047755 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.047861 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:16.050657 (+ 2796us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.65:40829: BlockingRecv error: recv got EOF from 127.18.71.65:40829 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":59,"threads_started":1}
Aug 09 07:49:16 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 07:49:16 dist-test-slave-10pc krb5kdc[27480](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.18.71.66@KRBTEST.COM for oryx/127.18.71.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 07:49:16.247520 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:16.240234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:34265 (local address 127.18.71.66:44651)
0809 07:49:16.240641 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:16.240660 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:16.240784 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:16.241213 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:16.241216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:16.241512 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:16.241646 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:16.241654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.242839 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.242842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:16.243764 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:16.243776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.244369 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.244373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:16.244507 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:16.245209 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:16.245228 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:16.247401 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.18.71.126:34265: Server oryx/127.18.71.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":90,"threads_started":1}
W20260809 07:49:16.247643 28651 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:16.240280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:44651 (local address 127.18.71.126:34265)
0809 07:49:16.240480 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:16.240487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:16.240884 (+ 397us) server_negotiation.cc:407] Connection header received
0809 07:49:16.240970 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:16.240975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:16.241059 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:16.241156 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:16.241779 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.242601 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.243896 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.244246 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.244330 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:16.247531 (+ 3201us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.18.71.66:44651: BlockingRecv error: recv got EOF from 127.18.71.66:44651 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:49:16.323637 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 28116
I20260809 07:49:16.336139 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 28287
I20260809 07:49:16.346423 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 28438
I20260809 07:49:16.353749 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 28040
2026-08-09T07:49:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18050 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T07:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:16Z Disabled control of system clock
I20260809 07:49:16.388166 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33277
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 07:49:16.501724 28656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:16.501991 28656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:16.502025 28656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:16.505937 28656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:16.505999 28656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:16.506016 28656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:16.506033 28656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:16.506048 28656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:16.510686 28656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:16.511732 28656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:16.513123 28656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:16.521600 28661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:16.521621 28664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:16.521602 28662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:16.521857 28656 server_base.cc:1034] running on GCE node
I20260809 07:49:16.522555 28656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:16.523597 28656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:16.524797 28656 hybrid_clock.cc:648] HybridClock initialized: now 1786261756524775 us; error 30 us; skew 500 ppm
I20260809 07:49:16.526800 28656 webserver.cc:460] Webserver started at http://127.18.71.126:45469/ using document root <none> and password file <none>
I20260809 07:49:16.527411 28656 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:16.527506 28656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:16.527760 28656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:16.529934 28656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "bfbc4a7949a74106908a18a785e7e0fe"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "829cb17b5a8ac62355f7e497dedd3361"
server_key_iv: "ff43019d5bb0c58544f3bafb6302e5cf"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.530532 28656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "bfbc4a7949a74106908a18a785e7e0fe"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "829cb17b5a8ac62355f7e497dedd3361"
server_key_iv: "ff43019d5bb0c58544f3bafb6302e5cf"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.534354 28656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:49:16.536780 28670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:16.538089 28656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:49:16.538237 28656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "bfbc4a7949a74106908a18a785e7e0fe"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "829cb17b5a8ac62355f7e497dedd3361"
server_key_iv: "ff43019d5bb0c58544f3bafb6302e5cf"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.538352 28656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:16.554575 28656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:16.555233 28656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:16.555438 28656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:16.562580 28656 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33277
I20260809 07:49:16.562585 28728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33277 every 8 connection(s)
I20260809 07:49:16.563558 28656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:16.564188 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 28656
I20260809 07:49:16.564301 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:49:16.564687 18717 external_mini_cluster.cc:1428] Setting key a8b69b5170a0ec097fddcebdf4f7194b
I20260809 07:49:16.567402 28729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:16.573239 28729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe: Bootstrap starting.
I20260809 07:49:16.574059 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:16.566163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.18.71.126:33277)
0809 07:49:16.566853 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:16.566868 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:16.566902 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:49:16.567612 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:16.567634 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:16.568019 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:16.568386 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:16.568838 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.569779 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.570705 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.571189 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.571361 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:16.572261 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:16.572266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:16.572278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:16.572714 (+ 436us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:16.572770 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:16.572791 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:16.572891 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:16.573065 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 07:49:16.573325 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":138,"threads_started":1}
I20260809 07:49:16.575811 28729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:16.576771 28729 log.cc:824] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:16.578881 28729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe: No bootstrap required, opened a new log
I20260809 07:49:16.582810 28729 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbc4a7949a74106908a18a785e7e0fe" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33277 } }
I20260809 07:49:16.583137 28729 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:16.583185 28729 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfbc4a7949a74106908a18a785e7e0fe, State: Initialized, Role: FOLLOWER
I20260809 07:49:16.583587 28729 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbc4a7949a74106908a18a785e7e0fe" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33277 } }
I20260809 07:49:16.583711 28729 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:49:16.583757 28729 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:49:16.583828 28729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:16.584868 28729 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbc4a7949a74106908a18a785e7e0fe" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33277 } }
I20260809 07:49:16.585254 28729 leader_election.cc:302] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfbc4a7949a74106908a18a785e7e0fe; no voters:
I20260809 07:49:16.585553 28729 leader_election.cc:288] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:49:16.585701 28734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:49:16.585963 28734 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [term 1 LEADER]: Becoming Leader. State: Replica: bfbc4a7949a74106908a18a785e7e0fe, State: Running, Role: LEADER
I20260809 07:49:16.586366 28734 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbc4a7949a74106908a18a785e7e0fe" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33277 } }
I20260809 07:49:16.587046 28729 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:16.588347 28736 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfbc4a7949a74106908a18a785e7e0fe. Latest consensus state: current_term: 1 leader_uuid: "bfbc4a7949a74106908a18a785e7e0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbc4a7949a74106908a18a785e7e0fe" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33277 } } }
I20260809 07:49:16.588331 28735 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfbc4a7949a74106908a18a785e7e0fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfbc4a7949a74106908a18a785e7e0fe" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33277 } } }
I20260809 07:49:16.588618 28736 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:16.588625 28735 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:16.589103 28744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:49:16.591928 28744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:49:16.597611 28744 catalog_manager.cc:1347] Generated new cluster ID: 615b3643c07b4271b9be40c0959c2a05
I20260809 07:49:16.597683 28744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:49:16.610603 28744 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:49:16.611855 28744 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:49:16.618999 28744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfbc4a7949a74106908a18a785e7e0fe: Generated new TSK 0
I20260809 07:49:16.619767 28744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:49:16.624006 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33277
--builtin_ntp_servers=127.18.71.84:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 07:49:16.738412 28753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:16.738656 28753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:16.738692 28753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:16.743042 28753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:16.743093 28753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:16.743146 28753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:49:16.747742 28753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:16.748754 28753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:16.750033 28753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:16.759611 28759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:16.759682 28760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:16.759665 28763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:16.759806 28753 server_base.cc:1034] running on GCE node
I20260809 07:49:16.760299 28753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:16.760939 28753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:16.762099 28753 hybrid_clock.cc:648] HybridClock initialized: now 1786261756762082 us; error 31 us; skew 500 ppm
I20260809 07:49:16.763916 28753 webserver.cc:460] Webserver started at http://127.18.71.65:42373/ using document root <none> and password file <none>
I20260809 07:49:16.764540 28753 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:16.764608 28753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:16.764847 28753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:16.766867 28753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "71db18aeb41141728e5b6768c42e97d5"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "d85c9ea0778fe432fd83f03d3308afdc"
server_key_iv: "e7392d657f5b90cffe1a1da108cddd34"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.767439 28753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "71db18aeb41141728e5b6768c42e97d5"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "d85c9ea0778fe432fd83f03d3308afdc"
server_key_iv: "e7392d657f5b90cffe1a1da108cddd34"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.771091 28753 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:49:16.773615 28771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:16.774798 28753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:49:16.774940 28753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "71db18aeb41141728e5b6768c42e97d5"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "d85c9ea0778fe432fd83f03d3308afdc"
server_key_iv: "e7392d657f5b90cffe1a1da108cddd34"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.775053 28753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:16.792346 28753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:16.793036 28753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:16.793238 28753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:16.793834 28753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:16.794770 28753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:16.794842 28753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:16.794917 28753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:16.794960 28753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:16.803936 28753 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:34351
I20260809 07:49:16.803992 28893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:34351 every 8 connection(s)
I20260809 07:49:16.804875 28753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:16.811261 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 28753
I20260809 07:49:16.811364 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:49:16.811733 18717 external_mini_cluster.cc:1428] Setting key f276b48a5da5ce18d7a9da17192285f6
I20260809 07:49:16.813927 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33277
--builtin_ntp_servers=127.18.71.84:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 07:49:16.815176 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:16.806843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:47819 (local address 127.18.71.126:33277)
0809 07:49:16.806999 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:16.807003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:16.808231 (+ 1228us) server_negotiation.cc:407] Connection header received
0809 07:49:16.809776 (+ 1545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:16.809779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:16.809829 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:16.809905 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:16.810837 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.811452 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.812911 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.813300 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.813416 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:16.814506 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:16.814514 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:16.814521 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:16.814540 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:16.814570 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:16.814586 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:16.814677 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:16.814842 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 07:49:16.814953 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:49:16.815800 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:16.807324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33277 (local address 127.18.71.65:47819)
0809 07:49:16.808076 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:16.808121 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:16.809578 (+ 1457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:16.810115 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:16.810122 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:16.810203 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:16.810708 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:16.810722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.811598 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.811601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:16.812781 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:16.812792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:16.813459 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:16.813462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:16.813607 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:16.814232 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:49:16.814234 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:16.814274 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:49:16.814336 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:16.814682 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:16.814689 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:16.814694 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:49:16.814798 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 07:49:16.815003 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":140,"threads_started":1}
I20260809 07:49:16.817072 28894 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33277
I20260809 07:49:16.817318 28894 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:16.817813 28894 heartbeater.cc:511] Master 127.18.71.126:33277 requested a full tablet report, sending...
I20260809 07:49:16.819293 28691 ts_manager.cc:194] Registered new tserver with Master: 71db18aeb41141728e5b6768c42e97d5 (127.18.71.65:34351)
I20260809 07:49:16.820411 28691 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.65:47819
W20260809 07:49:16.935570 28898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:16.935871 28898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:16.935943 28898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:16.940048 28898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:16.940145 28898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:16.940255 28898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:49:16.945477 28898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:16.946609 28898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:16.948045 28898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:49:16.958189 28898 server_base.cc:1034] running on GCE node
W20260809 07:49:16.958107 28903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:16.958122 28906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:16.958177 28904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:16.958889 28898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:16.959722 28898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:16.960915 28898 hybrid_clock.cc:648] HybridClock initialized: now 1786261756960897 us; error 35 us; skew 500 ppm
I20260809 07:49:16.962756 28898 webserver.cc:460] Webserver started at http://127.18.71.66:41865/ using document root <none> and password file <none>
I20260809 07:49:16.963359 28898 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:16.963429 28898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:16.963685 28898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:16.965838 28898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "52b8435810e341b6be78894761d30c4f"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "9673ec0a5db407e991d0c6ec5ad2d49e"
server_key_iv: "3b8556f1775ba28039f991367c542070"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.966435 28898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "52b8435810e341b6be78894761d30c4f"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "9673ec0a5db407e991d0c6ec5ad2d49e"
server_key_iv: "3b8556f1775ba28039f991367c542070"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.970278 28898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 07:49:16.972947 28913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:16.974247 28898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:49:16.974397 28898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "52b8435810e341b6be78894761d30c4f"
format_stamp: "Formatted at 2026-08-09 07:49:16 on dist-test-slave-10pc"
server_key: "9673ec0a5db407e991d0c6ec5ad2d49e"
server_key_iv: "3b8556f1775ba28039f991367c542070"
server_key_version: "encryptionkey@0"
I20260809 07:49:16.974525 28898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:16.991737 28898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:16.992547 28898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:16.992750 28898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:16.993364 28898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:16.994334 28898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:16.994410 28898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:16.994480 28898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:16.994530 28898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.003871 28898 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:45207
I20260809 07:49:17.003918 29039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:45207 every 8 connection(s)
I20260809 07:49:17.004879 28898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:17.012626 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 28898
I20260809 07:49:17.012760 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:49:17.013059 18717 external_mini_cluster.cc:1428] Setting key bc59c620779e2dc3bbfaecc670f8feb4
I20260809 07:49:17.014619 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.006860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:38335 (local address 127.18.71.126:33277)
0809 07:49:17.007034 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:17.007038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:17.008060 (+ 1022us) server_negotiation.cc:407] Connection header received
0809 07:49:17.009581 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:17.009584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:17.009634 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:17.009709 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:17.010655 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.011176 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.012316 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.012671 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.012796 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:17.014051 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:17.014055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:17.014058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:17.014077 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:17.014096 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:17.014104 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:17.014192 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:17.014403 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 07:49:17.014472 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:49:17.015239 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33277
--builtin_ntp_servers=127.18.71.84:44581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 07:49:17.015725 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.007206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33277 (local address 127.18.71.66:38335)
0809 07:49:17.007894 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:17.007936 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:17.009381 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:17.009881 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:17.009888 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:17.009972 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:17.010528 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:17.010542 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.011288 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.011291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:17.012186 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:17.012195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.012857 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.012868 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:17.013029 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:17.013757 (+ 728us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:49:17.013760 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:17.013811 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:49:17.013899 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:17.014195 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:17.014207 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:17.014214 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 07:49:17.014366 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 07:49:17.014663 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":124,"threads_started":1}
I20260809 07:49:17.016933 29040 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33277
I20260809 07:49:17.017196 29040 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:17.017732 29040 heartbeater.cc:511] Master 127.18.71.126:33277 requested a full tablet report, sending...
I20260809 07:49:17.018826 28691 ts_manager.cc:194] Registered new tserver with Master: 52b8435810e341b6be78894761d30c4f (127.18.71.66:45207)
I20260809 07:49:17.019665 28691 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.66:38335
W20260809 07:49:17.133479 29043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:17.133749 29043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:17.133785 29043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:17.137876 29043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:17.137943 29043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:17.137995 29043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:49:17.143013 29043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:17.144234 29043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:17.145845 29043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:17.156431 29049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:17.156565 29048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:17.156659 29043 server_base.cc:1034] running on GCE node
W20260809 07:49:17.156488 29051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:17.157186 29043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:17.157845 29043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:17.159055 29043 hybrid_clock.cc:648] HybridClock initialized: now 1786261757159027 us; error 39 us; skew 500 ppm
I20260809 07:49:17.161011 29043 webserver.cc:460] Webserver started at http://127.18.71.67:42729/ using document root <none> and password file <none>
I20260809 07:49:17.161614 29043 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:17.161679 29043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:17.161965 29043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:17.164113 29043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "f0004c6d73224793a780b84f9ffd4082"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "631bbde19ad5e42cf042e983dad0df14"
server_key_iv: "5c5694843a04dae69f0e7a9d09985f31"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.164731 29043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "f0004c6d73224793a780b84f9ffd4082"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "631bbde19ad5e42cf042e983dad0df14"
server_key_iv: "5c5694843a04dae69f0e7a9d09985f31"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.168569 29043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:49:17.171496 29058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.172902 29043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:49:17.173010 29043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "f0004c6d73224793a780b84f9ffd4082"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "631bbde19ad5e42cf042e983dad0df14"
server_key_iv: "5c5694843a04dae69f0e7a9d09985f31"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.173110 29043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:17.221480 29043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:17.222363 29043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:17.222597 29043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:17.223294 29043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:17.224612 29043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:17.224669 29043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.224740 29043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:17.224772 29043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.234892 29043 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:46197
I20260809 07:49:17.234915 29181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:46197 every 8 connection(s)
I20260809 07:49:17.235860 29043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:17.244648 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29043
I20260809 07:49:17.244792 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:17.245141 18717 external_mini_cluster.cc:1428] Setting key 493197cbb0ffce06da68c3a9f0faf53e
I20260809 07:49:17.245801 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.237874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:42439 (local address 127.18.71.126:33277)
0809 07:49:17.238032 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:17.238037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:17.239242 (+ 1205us) server_negotiation.cc:407] Connection header received
0809 07:49:17.240766 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:17.240769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:17.240818 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:17.240900 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:17.241871 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.242415 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.243697 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.243917 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.244019 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:17.245277 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:17.245280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:17.245282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:17.245295 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:17.245315 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:17.245323 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:17.245381 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:17.245582 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 07:49:17.245643 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:49:17.246840 29184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.238415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33277 (local address 127.18.71.67:42439)
0809 07:49:17.239087 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:17.239137 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:17.240418 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:17.241086 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:17.241093 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:17.241189 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:17.241738 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:17.241753 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.242520 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.242523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:17.243534 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:17.243564 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:17.244057 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:17.244059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:17.244213 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:17.244894 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:49:17.244897 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:17.244970 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:49:17.245058 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:17.245408 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:17.245418 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:17.245424 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 07:49:17.245561 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 07:49:17.245845 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
I20260809 07:49:17.247951 29182 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33277
I20260809 07:49:17.248207 29182 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:17.248795 29182 heartbeater.cc:511] Master 127.18.71.126:33277 requested a full tablet report, sending...
I20260809 07:49:17.249799 28691 ts_manager.cc:194] Registered new tserver with Master: f0004c6d73224793a780b84f9ffd4082 (127.18.71.67:46197)
I20260809 07:49:17.250316 28691 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.67:42439
I20260809 07:49:17.259431 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 07:49:17.262691 28732 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:17.261759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.18.71.126:33277)
0809 07:49:17.261907 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:17.261911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:17.261988 (+ 77us) server_negotiation.cc:407] Connection header received
0809 07:49:17.262102 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:17.262105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:17.262146 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:17.262213 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:17.262564 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43156: BlockingRecv error: recv got EOF from 127.0.0.1:43156 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 07:49:17.262894 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 28753
I20260809 07:49:17.272857 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 28898
I20260809 07:49:17.280629 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29043
I20260809 07:49:17.288113 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 28656
2026-08-09T07:49:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (932 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T07:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:17Z Disabled control of system clock
I20260809 07:49:17.320750 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45653
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:45653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 07:49:17.433784 29195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:17.434041 29195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:17.434077 29195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:17.437876 29195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:17.437920 29195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:17.437937 29195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:17.437952 29195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:17.437968 29195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:17.442546 29195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:45653
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:45653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:17.443488 29195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:17.444933 29195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:17.452883 29202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:17.452960 29204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:17.452884 29200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:17.453210 29195 server_base.cc:1034] running on GCE node
I20260809 07:49:17.453769 29195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:17.454799 29195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:17.455958 29195 hybrid_clock.cc:648] HybridClock initialized: now 1786261757455937 us; error 32 us; skew 500 ppm
I20260809 07:49:17.457729 29195 webserver.cc:460] Webserver started at http://127.18.71.126:43147/ using document root <none> and password file <none>
I20260809 07:49:17.458293 29195 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:17.458353 29195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:17.458602 29195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:17.460724 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "0cdd6c4d5acd43cda3100c411897e60b"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "e63dec0777803f4e5213652e50be2235"
server_key_iv: "70859a41388975682a677131830eeffd"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.461295 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "0cdd6c4d5acd43cda3100c411897e60b"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "e63dec0777803f4e5213652e50be2235"
server_key_iv: "70859a41388975682a677131830eeffd"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.464838 29195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:49:17.467172 29210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.468364 29195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:49:17.468513 29195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "0cdd6c4d5acd43cda3100c411897e60b"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "e63dec0777803f4e5213652e50be2235"
server_key_iv: "70859a41388975682a677131830eeffd"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.468613 29195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:17.473102 29195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:17.473688 29195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:17.473863 29195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:17.480669 29195 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:45653
I20260809 07:49:17.480671 29268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:45653 every 8 connection(s)
I20260809 07:49:17.481598 29195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:17.484992 29269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:17.486734 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29195
I20260809 07:49:17.486860 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:49:17.487167 18717 external_mini_cluster.cc:1428] Setting key cc17c62d5daa156478394f047a94081f
I20260809 07:49:17.491693 29269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b: Bootstrap starting.
I20260809 07:49:17.492810 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.488657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57758 (local address 127.18.71.126:45653)
0809 07:49:17.489297 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:17.489317 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:17.489351 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:49:17.489987 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:17.490003 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:17.490338 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:17.490673 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:17.491506 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:17.491510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:17.491522 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:17.491533 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:17.491589 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:17.491605 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:17.491756 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:17.491827 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 07:49:17.492058 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":135,"threads_started":1}
I20260809 07:49:17.494287 29269 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:17.495280 29269 log.cc:824] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:17.497236 29269 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b: No bootstrap required, opened a new log
I20260809 07:49:17.501169 29269 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdd6c4d5acd43cda3100c411897e60b" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45653 } }
I20260809 07:49:17.501407 29269 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:17.501441 29269 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cdd6c4d5acd43cda3100c411897e60b, State: Initialized, Role: FOLLOWER
I20260809 07:49:17.501799 29269 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdd6c4d5acd43cda3100c411897e60b" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45653 } }
I20260809 07:49:17.501906 29269 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:49:17.501950 29269 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:49:17.502017 29269 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:17.502936 29269 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdd6c4d5acd43cda3100c411897e60b" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45653 } }
I20260809 07:49:17.503252 29269 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cdd6c4d5acd43cda3100c411897e60b; no voters:
I20260809 07:49:17.503536 29269 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:49:17.503674 29275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:49:17.503929 29275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [term 1 LEADER]: Becoming Leader. State: Replica: 0cdd6c4d5acd43cda3100c411897e60b, State: Running, Role: LEADER
I20260809 07:49:17.504240 29275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdd6c4d5acd43cda3100c411897e60b" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45653 } }
I20260809 07:49:17.504870 29269 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:17.506109 29276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cdd6c4d5acd43cda3100c411897e60b. Latest consensus state: current_term: 1 leader_uuid: "0cdd6c4d5acd43cda3100c411897e60b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdd6c4d5acd43cda3100c411897e60b" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45653 } } }
I20260809 07:49:17.506134 29277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cdd6c4d5acd43cda3100c411897e60b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cdd6c4d5acd43cda3100c411897e60b" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 45653 } } }
I20260809 07:49:17.506258 29276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:17.506266 29277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:17.506717 29286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:49:17.509907 29286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:49:17.515209 29286 catalog_manager.cc:1347] Generated new cluster ID: 456c8201b5344646830a49f5f5710eae
I20260809 07:49:17.515275 29286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:49:17.529866 29286 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:49:17.530197 29286 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:49:17.537838 29286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cdd6c4d5acd43cda3100c411897e60b: Generated new TSK 0
I20260809 07:49:17.538544 29286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:49:17.549611 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45653
--builtin_ntp_servers=127.18.71.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 07:49:17.665220 29297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:17.665481 29297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:17.665558 29297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:17.669719 29297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:17.669778 29297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:17.669865 29297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:49:17.674765 29297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:17.675836 29297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:17.677343 29297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:49:17.687247 29297 server_base.cc:1034] running on GCE node
W20260809 07:49:17.687203 29303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:17.687197 29305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:17.687197 29302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:17.687789 29297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:17.688446 29297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:17.689728 29297 hybrid_clock.cc:648] HybridClock initialized: now 1786261757689713 us; error 26 us; skew 500 ppm
I20260809 07:49:17.691636 29297 webserver.cc:460] Webserver started at http://127.18.71.65:45721/ using document root <none> and password file <none>
I20260809 07:49:17.692224 29297 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:17.692317 29297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:17.692623 29297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:17.694665 29297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "dbc136fcc616483988cabd6ae171a645"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "b6e3da3d8bfd5c056feb0de000b9d809"
server_key_iv: "3fb04b5a343ab97b0db03b5c51a4057b"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.695250 29297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "dbc136fcc616483988cabd6ae171a645"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "b6e3da3d8bfd5c056feb0de000b9d809"
server_key_iv: "3fb04b5a343ab97b0db03b5c51a4057b"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.698983 29297 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:49:17.701550 29311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.702752 29297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:49:17.702885 29297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "dbc136fcc616483988cabd6ae171a645"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "b6e3da3d8bfd5c056feb0de000b9d809"
server_key_iv: "3fb04b5a343ab97b0db03b5c51a4057b"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.703001 29297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:17.707880 29297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:17.708530 29297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:17.708706 29297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:17.709237 29297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:17.710119 29297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:17.710196 29297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.710259 29297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:17.710311 29297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.719056 29297 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:36251
I20260809 07:49:17.719079 29433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:36251 every 8 connection(s)
I20260809 07:49:17.719911 29297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:17.726006 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.721717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:56805 (local address 127.18.71.126:45653)
0809 07:49:17.721854 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:17.721858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:17.722900 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 07:49:17.724204 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:17.724207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:17.724245 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:17.724323 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:17.725577 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:17.725580 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:17.725582 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:17.725594 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:17.725613 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:17.725623 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:17.725698 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:17.725870 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 07:49:17.725905 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:49:17.726889 29436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.722094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45653 (local address 127.18.71.65:56805)
0809 07:49:17.722750 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:17.722790 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:17.723970 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:17.724600 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:17.724607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:17.724696 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:17.725258 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:49:17.725260 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:17.725313 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:49:17.725413 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:17.725711 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:17.725721 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:17.725727 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 07:49:17.725906 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 07:49:17.726148 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":109,"threads_started":1}
I20260809 07:49:17.727344 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29297
I20260809 07:49:17.727471 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:49:17.727818 18717 external_mini_cluster.cc:1428] Setting key 9cc9f017a1d7762f45c127ca2a93f223
I20260809 07:49:17.728260 29434 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45653
I20260809 07:49:17.728561 29434 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:17.729019 29434 heartbeater.cc:511] Master 127.18.71.126:45653 requested a full tablet report, sending...
I20260809 07:49:17.729928 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45653
--builtin_ntp_servers=127.18.71.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 07:49:17.730645 29227 ts_manager.cc:194] Registered new tserver with Master: dbc136fcc616483988cabd6ae171a645 (127.18.71.65:36251)
W20260809 07:49:17.843700 29437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:17.843964 29437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:17.844005 29437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:17.848302 29437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:17.848371 29437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:17.848486 29437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:49:17.853225 29437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:17.854307 29437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:17.855736 29437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:17.865696 29444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:17.865734 29437 server_base.cc:1034] running on GCE node
W20260809 07:49:17.865722 29447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:17.865901 29442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:17.866325 29437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:17.867039 29437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:17.868233 29437 hybrid_clock.cc:648] HybridClock initialized: now 1786261757868209 us; error 34 us; skew 500 ppm
I20260809 07:49:17.870100 29437 webserver.cc:460] Webserver started at http://127.18.71.66:39043/ using document root <none> and password file <none>
I20260809 07:49:17.870682 29437 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:17.870746 29437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:17.871016 29437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:17.873152 29437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "b90a3a6119c44ade91ee6ba183c90187"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "a1b56d8dc8f22a68aaf211aaf25d66af"
server_key_iv: "d66958d2e43c0fa601728b1e5065c1d1"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.873720 29437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "b90a3a6119c44ade91ee6ba183c90187"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "a1b56d8dc8f22a68aaf211aaf25d66af"
server_key_iv: "d66958d2e43c0fa601728b1e5065c1d1"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.877348 29437 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:49:17.879793 29453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.880997 29437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:49:17.881161 29437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b90a3a6119c44ade91ee6ba183c90187"
format_stamp: "Formatted at 2026-08-09 07:49:17 on dist-test-slave-10pc"
server_key: "a1b56d8dc8f22a68aaf211aaf25d66af"
server_key_iv: "d66958d2e43c0fa601728b1e5065c1d1"
server_key_version: "encryptionkey@0"
I20260809 07:49:17.881273 29437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:17.885787 29437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:17.886379 29437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:17.886562 29437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:17.887105 29437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:17.887984 29437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:17.888051 29437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.888113 29437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:17.888152 29437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:17.897265 29437 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:38733
I20260809 07:49:17.897291 29573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:38733 every 8 connection(s)
I20260809 07:49:17.898138 29437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:17.904210 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.899910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:50707 (local address 127.18.71.126:45653)
0809 07:49:17.900045 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:17.900049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:17.901138 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 07:49:17.902472 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:17.902475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:17.902515 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:17.902607 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:17.903730 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:17.903732 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:17.903734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:17.903753 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:17.903775 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:17.903784 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:17.903864 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:17.904046 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 07:49:17.904092 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:49:17.905130 29576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:17.900284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45653 (local address 127.18.71.66:50707)
0809 07:49:17.901007 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:17.901047 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:17.902248 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:17.902774 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:17.902780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:17.902869 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:17.903415 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:49:17.903417 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:17.903475 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:49:17.903568 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:17.903861 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:17.903869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:17.903873 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 07:49:17.904004 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 07:49:17.904193 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":193,"threads_started":1}
I20260809 07:49:17.906391 29574 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45653
I20260809 07:49:17.906598 29574 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:17.907076 29574 heartbeater.cc:511] Master 127.18.71.126:45653 requested a full tablet report, sending...
I20260809 07:49:17.907208 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29437
I20260809 07:49:17.907330 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:49:17.907662 18717 external_mini_cluster.cc:1428] Setting key 8b9f47a7e2d8004280d83b80d8774c85
I20260809 07:49:17.908392 29227 ts_manager.cc:194] Registered new tserver with Master: b90a3a6119c44ade91ee6ba183c90187 (127.18.71.66:38733)
I20260809 07:49:17.909776 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45653
--builtin_ntp_servers=127.18.71.84:34219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 07:49:18.025107 29577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:18.025367 29577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:18.025408 29577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:18.029366 29577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:18.029417 29577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:18.029479 29577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:49:18.034091 29577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:45653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:18.035136 29577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:18.036621 29577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:18.046635 29582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:18.046707 29583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:18.046476 29587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:18.047073 29577 server_base.cc:1034] running on GCE node
I20260809 07:49:18.047472 29577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:18.048089 29577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:18.049271 29577 hybrid_clock.cc:648] HybridClock initialized: now 1786261758049252 us; error 31 us; skew 500 ppm
I20260809 07:49:18.051127 29577 webserver.cc:460] Webserver started at http://127.18.71.67:35341/ using document root <none> and password file <none>
I20260809 07:49:18.051717 29577 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:18.051781 29577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:18.052007 29577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:18.054171 29577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "2f6070cde2234e8c8dce335014783c0d"
format_stamp: "Formatted at 2026-08-09 07:49:18 on dist-test-slave-10pc"
server_key: "5d42ef12450c3a6bbd38ee8fcac708e0"
server_key_iv: "ea8d94675bbe82ccd1e343ec9b960141"
server_key_version: "encryptionkey@0"
I20260809 07:49:18.054765 29577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "2f6070cde2234e8c8dce335014783c0d"
format_stamp: "Formatted at 2026-08-09 07:49:18 on dist-test-slave-10pc"
server_key: "5d42ef12450c3a6bbd38ee8fcac708e0"
server_key_iv: "ea8d94675bbe82ccd1e343ec9b960141"
server_key_version: "encryptionkey@0"
I20260809 07:49:18.058660 29577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:49:18.061280 29594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:18.062572 29577 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:49:18.062708 29577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "2f6070cde2234e8c8dce335014783c0d"
format_stamp: "Formatted at 2026-08-09 07:49:18 on dist-test-slave-10pc"
server_key: "5d42ef12450c3a6bbd38ee8fcac708e0"
server_key_iv: "ea8d94675bbe82ccd1e343ec9b960141"
server_key_version: "encryptionkey@0"
I20260809 07:49:18.062825 29577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:18.067387 29577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:18.067981 29577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:18.068169 29577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:18.068775 29577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:18.069671 29577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:18.069742 29577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:18.069813 29577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:18.069862 29577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:18.079422 29577 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:39625
I20260809 07:49:18.079447 29715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:39625 every 8 connection(s)
I20260809 07:49:18.080335 29577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:18.086845 29273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:18.082292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:60785 (local address 127.18.71.126:45653)
0809 07:49:18.082435 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:18.082439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:18.083654 (+ 1215us) server_negotiation.cc:407] Connection header received
0809 07:49:18.085221 (+ 1567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:18.085224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:18.085263 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:18.085338 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:18.086463 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:18.086466 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:18.086467 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:49:18.086476 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:18.086493 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:49:18.086500 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:18.086573 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:18.086697 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 07:49:18.086736 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:49:18.087713 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29577
I20260809 07:49:18.087831 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:18.087868 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:18.082708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:45653 (local address 127.18.71.67:60785)
0809 07:49:18.083471 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:18.083529 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:18.084933 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:18.085480 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:18.085487 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:18.085601 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:18.086212 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:49:18.086214 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:18.086257 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:49:18.086328 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:18.086577 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:18.086584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:18.086589 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:49:18.086722 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 07:49:18.086915 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":110,"threads_started":1}
I20260809 07:49:18.088174 18717 external_mini_cluster.cc:1428] Setting key 7768c5386f2610419712c4a5e0ed22ca
I20260809 07:49:18.089197 29716 heartbeater.cc:347] Connected to a master server at 127.18.71.126:45653
I20260809 07:49:18.089468 29716 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:18.089891 29716 heartbeater.cc:511] Master 127.18.71.126:45653 requested a full tablet report, sending...
I20260809 07:49:18.090915 29227 ts_manager.cc:194] Registered new tserver with Master: 2f6070cde2234e8c8dce335014783c0d (127.18.71.67:39625)
I20260809 07:49:18.102578 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 07:49:18.105729 29273 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:49:18.104700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57764 (local address 127.18.71.126:45653)
0809 07:49:18.104832 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:18.104839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:18.105050 (+ 211us) server_negotiation.cc:407] Connection header received
0809 07:49:18.105207 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:18.105210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:18.105256 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:18.105354 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:18.105583 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57764: BlockingRecv error: recv got EOF from 127.0.0.1:57764 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:49:18.105866 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29297
I20260809 07:49:18.119020 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29437
I20260809 07:49:18.127137 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29577
I20260809 07:49:18.134619 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29195
2026-08-09T07:49:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (847 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:49:18 dist-test-slave-10pc krb5kdc[29727](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:49:18 dist-test-slave-10pc krb5kdc[29727](info): set up 2 sockets
Aug 09 07:49:18 dist-test-slave-10pc krb5kdc[29727](info): commencing operation
krb5kdc: starting...
W20260809 07:49:19.225464 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:49:19 dist-test-slave-10pc krb5kdc[29727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:49:19.360427 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:41729
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:41729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:49:19.475752 29741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:19.476047 29741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:19.476122 29741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:19.480095 29741 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:49:19.480187 29741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:19.480225 29741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:19.480283 29741 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:49:19.480324 29741 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:49:19.485530 29741 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:41729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:41729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:19.486564 29741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:19.487613 29741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:19.496049 29750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:19.496065 29748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:19.496055 29747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:19.496416 29741 server_base.cc:1034] running on GCE node
I20260809 07:49:19.497041 29741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:19.498121 29741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:19.499292 29741 hybrid_clock.cc:648] HybridClock initialized: now 1786261759499262 us; error 40 us; skew 500 ppm
Aug 09 07:49:19 dist-test-slave-10pc krb5kdc[29727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:19.502125 29741 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:49:19.503042 29741 webserver.cc:460] Webserver started at http://127.18.71.126:41009/ using document root <none> and password file <none>
I20260809 07:49:19.503625 29741 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:19.503718 29741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:19.503960 29741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:19.506057 29741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "6df9771d35d340338e325de221436f6a"
format_stamp: "Formatted at 2026-08-09 07:49:19 on dist-test-slave-10pc"
server_key: "22de9cb7ce665cd5e0214e200dcda66a"
server_key_iv: "137f6e5a4c09100d4008ee1edff2dbb0"
server_key_version: "encryptionkey@0"
I20260809 07:49:19.506631 29741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "6df9771d35d340338e325de221436f6a"
format_stamp: "Formatted at 2026-08-09 07:49:19 on dist-test-slave-10pc"
server_key: "22de9cb7ce665cd5e0214e200dcda66a"
server_key_iv: "137f6e5a4c09100d4008ee1edff2dbb0"
server_key_version: "encryptionkey@0"
I20260809 07:49:19.510367 29741 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:49:19.512842 29757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:19.514171 29741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:49:19.514317 29741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "6df9771d35d340338e325de221436f6a"
format_stamp: "Formatted at 2026-08-09 07:49:19 on dist-test-slave-10pc"
server_key: "22de9cb7ce665cd5e0214e200dcda66a"
server_key_iv: "137f6e5a4c09100d4008ee1edff2dbb0"
server_key_version: "encryptionkey@0"
I20260809 07:49:19.514431 29741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:19.530433 29741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:19.533875 29741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:19.534080 29741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:19.541294 29741 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:41729
I20260809 07:49:19.541599 29814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:41729 every 8 connection(s)
I20260809 07:49:19.542286 29741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:49:19.545151 29815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:19.547120 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29741
I20260809 07:49:19.547267 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:49:19.547544 18717 external_mini_cluster.cc:1428] Setting key 08f4b69de44c76ffca0b640a27e78c40
I20260809 07:49:19.551251 29815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a: Bootstrap starting.
I20260809 07:49:19.553836 29815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:19.554647 29815 log.cc:824] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:49:19 dist-test-slave-10pc krb5kdc[29727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:49:19.556850 29815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a: No bootstrap required, opened a new log
I20260809 07:49:19.561373 29815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df9771d35d340338e325de221436f6a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41729 } }
I20260809 07:49:19.561587 29815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:19.561676 29815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6df9771d35d340338e325de221436f6a, State: Initialized, Role: FOLLOWER
I20260809 07:49:19.562152 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:19.549042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.18.71.126:41729)
0809 07:49:19.549464 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:19.549479 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:19.549515 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:49:19.550305 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:19.550325 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:19.550651 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:19.550928 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:19.551860 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:19.552921 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:19.554071 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:19.554558 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:19.554680 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:19.557485 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:19.557490 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:19.557502 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:19.557528 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:19.559210 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:19.559701 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:19.559706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:19.559710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:19.559813 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:19.560092 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:19.560094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:19.560095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:19.560532 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:19.560694 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:19.560986 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 07:49:19.561309 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":141,"threads_started":1}
I20260809 07:49:19.562124 29815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df9771d35d340338e325de221436f6a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41729 } }
I20260809 07:49:19.562322 29815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:49:19.562422 29815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:49:19.562561 29815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:19.563431 29815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df9771d35d340338e325de221436f6a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41729 } }
I20260809 07:49:19.563776 29815 leader_election.cc:302] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6df9771d35d340338e325de221436f6a; no voters:
I20260809 07:49:19.564116 29815 leader_election.cc:288] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:49:19.564193 29820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:49:19.564404 29820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [term 1 LEADER]: Becoming Leader. State: Replica: 6df9771d35d340338e325de221436f6a, State: Running, Role: LEADER
I20260809 07:49:19.564738 29820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df9771d35d340338e325de221436f6a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41729 } }
I20260809 07:49:19.565445 29815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:49:19.566481 29822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6df9771d35d340338e325de221436f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df9771d35d340338e325de221436f6a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41729 } } }
I20260809 07:49:19.566612 29822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:19.567026 29830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:49:19.566982 29821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6df9771d35d340338e325de221436f6a. Latest consensus state: current_term: 1 leader_uuid: "6df9771d35d340338e325de221436f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6df9771d35d340338e325de221436f6a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41729 } } }
I20260809 07:49:19.567713 29821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a [sys.catalog]: This master's current role is: LEADER
I20260809 07:49:19.569816 29830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:49:19.575037 29830 catalog_manager.cc:1347] Generated new cluster ID: fb84ac1969a042418808b4730bd01b99
I20260809 07:49:19.575110 29830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:49:19.613843 29830 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:49:19.614951 29830 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:49:19.626583 29830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a: Generated new TSK 0
I20260809 07:49:19.627429 29830 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:49:19.667968 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:41729
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:49:19.781759 29842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:19.782100 29842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:19.782228 29842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:19.786481 29842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:19.786586 29842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:19.786686 29842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:49:19.792008 29842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:19.793115 29842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:19.794162 29842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:19.803907 29848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:19.804077 29842 server_base.cc:1034] running on GCE node
W20260809 07:49:19.803902 29849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:19.803902 29851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:19.804706 29842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:19.805375 29842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:19.806542 29842 hybrid_clock.cc:648] HybridClock initialized: now 1786261759806519 us; error 33 us; skew 500 ppm
Aug 09 07:49:19 dist-test-slave-10pc krb5kdc[29727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:19.809316 29842 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:49:19.810212 29842 webserver.cc:460] Webserver started at http://127.18.71.65:44137/ using document root <none> and password file <none>
I20260809 07:49:19.810779 29842 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:19.810873 29842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:19.811115 29842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:19.813258 29842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "048c473f6d694ba381267c3b56e0f89e"
format_stamp: "Formatted at 2026-08-09 07:49:19 on dist-test-slave-10pc"
server_key: "4202164aa73c0d12a2cd57c9731f3281"
server_key_iv: "4b4c4c5a2c1a626662712350067d2065"
server_key_version: "encryptionkey@0"
I20260809 07:49:19.813844 29842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "048c473f6d694ba381267c3b56e0f89e"
format_stamp: "Formatted at 2026-08-09 07:49:19 on dist-test-slave-10pc"
server_key: "4202164aa73c0d12a2cd57c9731f3281"
server_key_iv: "4b4c4c5a2c1a626662712350067d2065"
server_key_version: "encryptionkey@0"
I20260809 07:49:19.818030 29842 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:49:19.820557 29858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:19.821746 29842 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:49:19.821880 29842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "048c473f6d694ba381267c3b56e0f89e"
format_stamp: "Formatted at 2026-08-09 07:49:19 on dist-test-slave-10pc"
server_key: "4202164aa73c0d12a2cd57c9731f3281"
server_key_iv: "4b4c4c5a2c1a626662712350067d2065"
server_key_version: "encryptionkey@0"
I20260809 07:49:19.821988 29842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:19.833107 29842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:19.835788 29842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:19.835994 29842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:19.836643 29842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:19.837482 29842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:19.837558 29842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:19.837627 29842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:19.837689 29842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:19.846447 29842 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:37041
I20260809 07:49:19.846470 29985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:37041 every 8 connection(s)
I20260809 07:49:19.847338 29842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:49:19.855549 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29842
I20260809 07:49:19.855652 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:49:19.856042 18717 external_mini_cluster.cc:1428] Setting key 68283c608d16273888e77de3593518ab
Aug 09 07:49:19 dist-test-slave-10pc krb5kdc[29727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261759, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:49:19.862007 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:19.849204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:50785 (local address 127.18.71.126:41729)
0809 07:49:19.849373 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:19.849380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:19.850235 (+ 855us) server_negotiation.cc:407] Connection header received
0809 07:49:19.851782 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:19.851786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:19.851868 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:19.851970 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:19.853388 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:19.853919 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:19.855015 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:19.855269 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:19.855382 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:19.858338 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:19.858342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:19.858344 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:19.858365 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:19.859952 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:19.860491 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:19.860494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:19.860496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:19.860584 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:19.860937 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:19.860942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:19.860944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:19.861127 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:19.861241 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:19.861719 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 07:49:19.861843 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:49:19.862851 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:19.849642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:41729 (local address 127.18.71.65:50785)
0809 07:49:19.850080 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:19.850124 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:19.851511 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:19.852284 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:19.852294 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:19.852767 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:19.853250 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:19.853264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:19.854029 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:19.854031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:19.854910 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:19.854919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:19.855413 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:19.855416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:19.855552 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:19.856255 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:19.856270 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:19.858161 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:19.860081 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:19.860087 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:19.860101 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:19.860362 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:19.860689 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:19.860692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:19.860694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:19.860837 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:19.861239 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:19.861246 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:19.861487 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 07:49:19.861681 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 07:49:19.861992 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
I20260809 07:49:19.864071 29986 heartbeater.cc:347] Connected to a master server at 127.18.71.126:41729
I20260809 07:49:19.864321 29986 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:19.864835 29986 heartbeater.cc:511] Master 127.18.71.126:41729 requested a full tablet report, sending...
I20260809 07:49:19.866585 29775 ts_manager.cc:194] Registered new tserver with Master: 048c473f6d694ba381267c3b56e0f89e (127.18.71.65:37041)
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:49:19.867954 29775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:50785
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:49:19.884614 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:41729
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:49:19.999054 29991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:19.999308 29991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:19.999351 29991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:20.003651 29991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:20.003726 29991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:20.003783 29991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:49:20.009346 29991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:20.010495 29991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:20.011579 29991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:20.022351 29998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:20.022351 29997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:20.022544 29991 server_base.cc:1034] running on GCE node
W20260809 07:49:20.022351 30000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:20.022972 29991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:20.023640 29991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:20.024837 29991 hybrid_clock.cc:648] HybridClock initialized: now 1786261760024819 us; error 27 us; skew 500 ppm
Aug 09 07:49:20 dist-test-slave-10pc krb5kdc[29727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261760, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:20.027756 29991 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:49:20.028738 29991 webserver.cc:460] Webserver started at http://127.18.71.66:34683/ using document root <none> and password file <none>
I20260809 07:49:20.029280 29991 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:20.029341 29991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:20.029510 29991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:20.031556 29991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "c6dcd603691e41d7b13a636e2fa66328"
format_stamp: "Formatted at 2026-08-09 07:49:20 on dist-test-slave-10pc"
server_key: "db963ad354ff5b31710eb79f615a1f6f"
server_key_iv: "9207df9b89cbb0ec77418726795e9095"
server_key_version: "encryptionkey@0"
I20260809 07:49:20.032107 29991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "c6dcd603691e41d7b13a636e2fa66328"
format_stamp: "Formatted at 2026-08-09 07:49:20 on dist-test-slave-10pc"
server_key: "db963ad354ff5b31710eb79f615a1f6f"
server_key_iv: "9207df9b89cbb0ec77418726795e9095"
server_key_version: "encryptionkey@0"
I20260809 07:49:20.035936 29991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:49:20.038736 30007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:20.040050 29991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:49:20.040231 29991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "c6dcd603691e41d7b13a636e2fa66328"
format_stamp: "Formatted at 2026-08-09 07:49:20 on dist-test-slave-10pc"
server_key: "db963ad354ff5b31710eb79f615a1f6f"
server_key_iv: "9207df9b89cbb0ec77418726795e9095"
server_key_version: "encryptionkey@0"
I20260809 07:49:20.040352 29991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:20.056838 29991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:20.060256 29991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:20.060420 29991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:20.061115 29991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:20.062103 29991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:20.062151 29991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:20.062191 29991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:20.062211 29991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:20.072072 29991 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:37323
I20260809 07:49:20.072093 30131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:37323 every 8 connection(s)
I20260809 07:49:20.073088 29991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:49:20.081454 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 29991
I20260809 07:49:20.081614 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:49:20.081995 18717 external_mini_cluster.cc:1428] Setting key f1bc10f97ed5711b5b249db54b703545
Aug 09 07:49:20 dist-test-slave-10pc krb5kdc[29727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261760, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:49:20.088163 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.074963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:48839 (local address 127.18.71.126:41729)
0809 07:49:20.075117 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.075121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.076003 (+ 882us) server_negotiation.cc:407] Connection header received
0809 07:49:20.077583 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.077586 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.077642 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.077747 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:20.078970 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.079544 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.080796 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.081038 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.081155 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.083863 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:20.083867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:20.083870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:20.083896 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:20.085650 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:20.086349 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:20.086351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:20.086353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:20.086514 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:20.086892 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:20.086897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:20.086902 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:20.087178 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:20.087285 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.087814 (+ 529us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.087958 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 07:49:20.089133 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.075323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:41729 (local address 127.18.71.66:48839)
0809 07:49:20.075787 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.075851 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:20.077338 (+ 1487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:20.077927 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:20.077934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:20.078306 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:20.078811 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.078824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.079686 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.079689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.080666 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.080676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.081196 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.081201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.081378 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.082087 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:20.082114 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:20.083647 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:20.085778 (+ 2131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:20.085786 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:20.085806 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:20.086234 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:20.086631 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:20.086634 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:20.086636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:20.086794 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:20.087292 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:20.087298 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:20.087556 (+ 258us) client_negotiation.cc:769] Sending connection context
0809 07:49:20.087721 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 07:49:20.087982 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":100,"threads_started":1}
I20260809 07:49:20.090371 30132 heartbeater.cc:347] Connected to a master server at 127.18.71.126:41729
I20260809 07:49:20.090648 30132 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:20.091269 30132 heartbeater.cc:511] Master 127.18.71.126:41729 requested a full tablet report, sending...
I20260809 07:49:20.092442 29775 ts_manager.cc:194] Registered new tserver with Master: c6dcd603691e41d7b13a636e2fa66328 (127.18.71.66:37323)
I20260809 07:49:20.093124 29775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:48839
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:49:20.113196 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:41729
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:49:20.229357 30138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:49:20.229604 30138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:49:20.229674 30138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:49:20.233721 30138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:49:20.233779 30138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:49:20.233868 30138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:49:20.241092 30138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:41729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:49:20.242668 30138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:49:20.244091 30138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:49:20.253818 30147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:20.253832 30143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:49:20.253832 30144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:49:20.254027 30138 server_base.cc:1034] running on GCE node
I20260809 07:49:20.254426 30138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:49:20.254992 30138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:49:20.256184 30138 hybrid_clock.cc:648] HybridClock initialized: now 1786261760256156 us; error 46 us; skew 500 ppm
Aug 09 07:49:20 dist-test-slave-10pc krb5kdc[29727](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261760, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:49:20.258912 30138 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:49:20.259792 30138 webserver.cc:460] Webserver started at http://127.18.71.67:38163/ using document root <none> and password file <none>
I20260809 07:49:20.260361 30138 fs_manager.cc:359] Metadata directory not provided
I20260809 07:49:20.260454 30138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:49:20.260725 30138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:49:20.262744 30138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "af6271f465e94800b3908f22c258a9f0"
format_stamp: "Formatted at 2026-08-09 07:49:20 on dist-test-slave-10pc"
server_key: "d435727c7b341494036685a14b4e5b40"
server_key_iv: "0818c8ca213fa3f7ba38b28191adcf6a"
server_key_version: "encryptionkey@0"
I20260809 07:49:20.263329 30138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "af6271f465e94800b3908f22c258a9f0"
format_stamp: "Formatted at 2026-08-09 07:49:20 on dist-test-slave-10pc"
server_key: "d435727c7b341494036685a14b4e5b40"
server_key_iv: "0818c8ca213fa3f7ba38b28191adcf6a"
server_key_version: "encryptionkey@0"
I20260809 07:49:20.267314 30138 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 07:49:20.269891 30155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:20.271100 30138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:49:20.271245 30138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "af6271f465e94800b3908f22c258a9f0"
format_stamp: "Formatted at 2026-08-09 07:49:20 on dist-test-slave-10pc"
server_key: "d435727c7b341494036685a14b4e5b40"
server_key_iv: "0818c8ca213fa3f7ba38b28191adcf6a"
server_key_version: "encryptionkey@0"
I20260809 07:49:20.271360 30138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:49:20.288797 30138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:49:20.291822 30138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:49:20.292035 30138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:49:20.292651 30138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:49:20.293494 30138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:49:20.293568 30138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:20.293643 30138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:49:20.293690 30138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:49:20.302635 30138 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:45263
I20260809 07:49:20.302665 30276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:45263 every 8 connection(s)
I20260809 07:49:20.303575 30138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:49:20.310333 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 30138
I20260809 07:49:20.310472 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:49:20.310825 18717 external_mini_cluster.cc:1428] Setting key fe1f5856511e3ebe294caf8b6164716a
Aug 09 07:49:20 dist-test-slave-10pc krb5kdc[29727](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261760, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:49:20.317970 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.305508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:56637 (local address 127.18.71.126:41729)
0809 07:49:20.305646 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.305650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.306480 (+ 830us) server_negotiation.cc:407] Connection header received
0809 07:49:20.307890 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.307893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.307951 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.308053 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:20.309308 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.309842 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.311440 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.311669 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.311781 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.314380 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:20.314384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:20.314387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:20.314417 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:20.315880 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:20.316396 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:20.316404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:20.316405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:20.316525 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:20.316870 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:20.316875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:20.316876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:20.317061 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:20.317171 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.317666 (+ 495us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.317791 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:49:20.318742 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.305880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:41729 (local address 127.18.71.67:56637)
0809 07:49:20.306334 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.306375 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:20.307680 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:20.308233 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:20.308240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:20.308680 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:49:20.309158 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.309175 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.309969 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.309973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.311311 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.311320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.311819 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.311824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.311997 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.312649 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:49:20.312665 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:49:20.314220 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:49:20.315999 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:20.316005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:20.316020 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:20.316285 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:20.316613 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:49:20.316616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:49:20.316621 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:49:20.316763 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:49:20.317172 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:49:20.317180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:49:20.317425 (+ 245us) client_negotiation.cc:769] Sending connection context
0809 07:49:20.317612 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 07:49:20.317938 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":115,"threads_started":1}
I20260809 07:49:20.319831 30277 heartbeater.cc:347] Connected to a master server at 127.18.71.126:41729
I20260809 07:49:20.320062 30277 heartbeater.cc:464] Registering TS with master...
I20260809 07:49:20.320636 30277 heartbeater.cc:511] Master 127.18.71.126:41729 requested a full tablet report, sending...
I20260809 07:49:20.321622 29775 ts_manager.cc:194] Registered new tserver with Master: af6271f465e94800b3908f22c258a9f0 (127.18.71.67:45263)
I20260809 07:49:20.322196 29775 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:56637
I20260809 07:49:20.324867 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:49:20.333899 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.326958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49160 (local address 127.18.71.126:41729)
0809 07:49:20.327081 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.327085 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.327274 (+ 189us) server_negotiation.cc:407] Connection header received
0809 07:49:20.327362 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.327364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.327400 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.327460 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:49:20.328104 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.328660 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.329460 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.329671 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.329743 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.331011 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:49:20.331013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:49:20.331015 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:49:20.331036 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:49:20.332498 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:20.332897 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:20.332900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:20.332901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:20.332977 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:49:20.333205 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:49:20.333207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:49:20.333209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:49:20.333415 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:49:20.333516 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.333691 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.333782 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:49:20.336795 29775 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:49:20.339857 29775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:49:20.354508 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.349194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44802 (local address 127.18.71.67:45263)
0809 07:49:20.349985 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.349997 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.350020 (+ 23us) server_negotiation.cc:407] Connection header received
0809 07:49:20.350089 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.350093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.350260 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.350805 (+ 545us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:49:20.350958 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.351706 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.353231 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.353815 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.353940 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.354093 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.354255 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.354347 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686,"thread_start_us":143,"threads_started":1}
I20260809 07:49:20.354501 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.348709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:45263 (local address 127.0.0.1:44802)
0809 07:49:20.349602 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.349616 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:20.349706 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:20.350474 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:20.350483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:20.350506 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:49:20.350654 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.350662 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.351840 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.351846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.353107 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.353117 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.353942 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.353946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.354052 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.354087 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 07:49:20.354224 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 07:49:20.354347 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"thread_start_us":115,"threads_started":1}
I20260809 07:49:20.355311 30291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.348977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:37323 (local address 127.0.0.1:54000)
0809 07:49:20.349929 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.349942 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:20.350030 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:20.350683 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:20.350686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:20.350715 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:49:20.350874 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.350880 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.352356 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.352359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.353842 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.353854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.354956 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.354959 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.355041 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.355063 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 07:49:20.355123 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 07:49:20.355184 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":139,"spinlock_wait_cycles":3456,"thread_start_us":104,"threads_started":1}
I20260809 07:49:20.355407 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.349383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.18.71.66:37323)
0809 07:49:20.349903 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.349915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.350005 (+ 90us) server_negotiation.cc:407] Connection header received
0809 07:49:20.350137 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.350142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.350338 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.350485 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:49:20.351031 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.352199 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.353990 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.354822 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.354949 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.355086 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.355153 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.355245 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":126,"threads_started":1}
I20260809 07:49:20.355590 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.348573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37041 (local address 127.0.0.1:40318)
0809 07:49:20.349509 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.349565 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:20.349694 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:20.350675 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:20.350678 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:20.350699 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:49:20.350849 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.350858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.352356 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.352359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.353842 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.353854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.355215 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.355219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.355311 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.355337 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:49:20.355386 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 07:49:20.355458 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":141,"spinlock_wait_cycles":768,"thread_start_us":85,"threads_started":1}
I20260809 07:49:20.358013 30204 tablet_service.cc:1467] Processing CreateTablet for tablet 2544e7a5cee147c1a6cfa2cf454df62e (DEFAULT_TABLE table=test-table [id=a66cbdc268904e81aa982ad0666a5485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:49:20.358450 30059 tablet_service.cc:1467] Processing CreateTablet for tablet 2544e7a5cee147c1a6cfa2cf454df62e (DEFAULT_TABLE table=test-table [id=a66cbdc268904e81aa982ad0666a5485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:49:20.359238 30204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2544e7a5cee147c1a6cfa2cf454df62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:20.359479 30059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2544e7a5cee147c1a6cfa2cf454df62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:20.364182 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.348701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40318 (local address 127.18.71.65:37041)
0809 07:49:20.350264 (+ 1563us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.350281 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.350302 (+ 21us) server_negotiation.cc:407] Connection header received
0809 07:49:20.350373 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.350378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.350546 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.350685 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:49:20.350976 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.352191 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.353990 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.355067 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.363811 (+ 8744us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.363936 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.363982 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.364068 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1457,"thread_start_us":85,"threads_started":1}
I20260809 07:49:20.367004 30295 tablet_bootstrap.cc:492] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Bootstrap starting.
I20260809 07:49:20.367033 29912 tablet_service.cc:1467] Processing CreateTablet for tablet 2544e7a5cee147c1a6cfa2cf454df62e (DEFAULT_TABLE table=test-table [id=a66cbdc268904e81aa982ad0666a5485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:49:20.367848 30296 tablet_bootstrap.cc:492] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Bootstrap starting.
I20260809 07:49:20.368156 29912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2544e7a5cee147c1a6cfa2cf454df62e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:49:20.370419 30295 tablet_bootstrap.cc:654] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:20.370661 30296 tablet_bootstrap.cc:654] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:20.371321 30295 log.cc:824] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:20.371506 30296 log.cc:824] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:20.373339 30296 tablet_bootstrap.cc:492] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: No bootstrap required, opened a new log
I20260809 07:49:20.373528 30295 tablet_bootstrap.cc:492] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: No bootstrap required, opened a new log
I20260809 07:49:20.373590 30296 ts_tablet_manager.cc:1397] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:49:20.373744 30295 ts_tablet_manager.cc:1397] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:49:20.375793 30299 tablet_bootstrap.cc:492] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: Bootstrap starting.
I20260809 07:49:20.377444 30296 raft_consensus.cc:348] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.377447 30295 raft_consensus.cc:348] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.377733 30296 raft_consensus.cc:374] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:20.377733 30295 raft_consensus.cc:374] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:20.377791 30296 raft_consensus.cc:729] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af6271f465e94800b3908f22c258a9f0, State: Initialized, Role: FOLLOWER
I20260809 07:49:20.377805 30295 raft_consensus.cc:729] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6dcd603691e41d7b13a636e2fa66328, State: Initialized, Role: FOLLOWER
I20260809 07:49:20.378260 30295 consensus_queue.cc:260] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.378240 30296 consensus_queue.cc:260] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.378574 30299 tablet_bootstrap.cc:654] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: Neither blocks nor log segments found. Creating new log.
I20260809 07:49:20.379031 30277 heartbeater.cc:503] Master 127.18.71.126:41729 was elected leader, sending a full tablet report...
I20260809 07:49:20.379287 30132 heartbeater.cc:503] Master 127.18.71.126:41729 was elected leader, sending a full tablet report...
I20260809 07:49:20.379319 30296 ts_tablet_manager.cc:1428] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:49:20.379690 30295 ts_tablet_manager.cc:1428] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:49:20.380640 30299 log.cc:824] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: Log is configured to *not* fsync() on all Append() calls
I20260809 07:49:20.382912 30299 tablet_bootstrap.cc:492] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: No bootstrap required, opened a new log
I20260809 07:49:20.383126 30299 ts_tablet_manager.cc:1397] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260809 07:49:20.387076 30299 raft_consensus.cc:348] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.387296 30299 raft_consensus.cc:374] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:49:20.387373 30299 raft_consensus.cc:729] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048c473f6d694ba381267c3b56e0f89e, State: Initialized, Role: FOLLOWER
I20260809 07:49:20.387777 30299 consensus_queue.cc:260] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.388415 29986 heartbeater.cc:503] Master 127.18.71.126:41729 was elected leader, sending a full tablet report...
I20260809 07:49:20.388690 30299 ts_tablet_manager.cc:1428] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
W20260809 07:49:20.554759 30278 tablet.cc:2369] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:49:20.574429 30133 tablet.cc:2369] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:49:20.581895 30300 raft_consensus.cc:482] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:49:20.582139 30300 raft_consensus.cc:504] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.583329 30300 leader_election.cc:288] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 048c473f6d694ba381267c3b56e0f89e (127.18.71.65:37041), af6271f465e94800b3908f22c258a9f0 (127.18.71.67:45263)
I20260809 07:49:20.587090 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.583573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37041 (local address 127.18.71.66:39407)
0809 07:49:20.583753 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.583771 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:20.583868 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:20.584137 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:20.584140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:20.584160 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:49:20.584287 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.584293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.585091 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.585095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.585950 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.585960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.586636 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.586640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.586757 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.586776 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 07:49:20.586868 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 07:49:20.586943 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":41}
I20260809 07:49:20.587090 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.583734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:39407 (local address 127.18.71.65:37041)
0809 07:49:20.583874 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.583878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.583894 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:49:20.583968 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.583973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.584022 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.584109 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:49:20.584416 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.584964 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.586084 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.586502 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.586636 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.586691 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.586896 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.586970 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:49:20.587374 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.583697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:45391 (local address 127.18.71.67:45263)
0809 07:49:20.583821 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.583826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.584209 (+ 383us) server_negotiation.cc:407] Connection header received
0809 07:49:20.584367 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.584372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.584422 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.584538 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:49:20.584829 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.585341 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.586409 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.586807 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.586943 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.586992 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.587190 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.587259 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 07:49:20.587395 30304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.583573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:45263 (local address 127.18.71.66:45391)
0809 07:49:20.584088 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.584100 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:49:20.584238 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:49:20.584561 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:49:20.584564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:49:20.584581 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:49:20.584716 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.584722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.585457 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.585462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.586280 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:49:20.586289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.586938 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.586941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:49:20.587049 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.587070 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:49:20.587164 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 07:49:20.587238 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":212,"thread_start_us":120,"threads_started":1}
I20260809 07:49:20.587806 29935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2544e7a5cee147c1a6cfa2cf454df62e" candidate_uuid: "c6dcd603691e41d7b13a636e2fa66328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048c473f6d694ba381267c3b56e0f89e" is_pre_election: true
I20260809 07:49:20.588014 30224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2544e7a5cee147c1a6cfa2cf454df62e" candidate_uuid: "c6dcd603691e41d7b13a636e2fa66328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6271f465e94800b3908f22c258a9f0" is_pre_election: true
I20260809 07:49:20.588155 29935 raft_consensus.cc:2454] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6dcd603691e41d7b13a636e2fa66328 in term 0.
I20260809 07:49:20.588335 30224 raft_consensus.cc:2454] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6dcd603691e41d7b13a636e2fa66328 in term 0.
I20260809 07:49:20.588768 30010 leader_election.cc:302] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048c473f6d694ba381267c3b56e0f89e, c6dcd603691e41d7b13a636e2fa66328; no voters:
I20260809 07:49:20.589087 30300 raft_consensus.cc:2781] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:49:20.589184 30300 raft_consensus.cc:482] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:49:20.589228 30300 raft_consensus.cc:3037] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:20.590272 30300 raft_consensus.cc:504] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.590626 30300 leader_election.cc:288] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [CANDIDATE]: Term 1 election: Requested vote from peers 048c473f6d694ba381267c3b56e0f89e (127.18.71.65:37041), af6271f465e94800b3908f22c258a9f0 (127.18.71.67:45263)
I20260809 07:49:20.590965 29935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2544e7a5cee147c1a6cfa2cf454df62e" candidate_uuid: "c6dcd603691e41d7b13a636e2fa66328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048c473f6d694ba381267c3b56e0f89e"
I20260809 07:49:20.591039 30224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2544e7a5cee147c1a6cfa2cf454df62e" candidate_uuid: "c6dcd603691e41d7b13a636e2fa66328" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af6271f465e94800b3908f22c258a9f0"
I20260809 07:49:20.591104 29935 raft_consensus.cc:3037] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:20.591141 30224 raft_consensus.cc:3037] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:49:20.592485 29935 raft_consensus.cc:2454] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6dcd603691e41d7b13a636e2fa66328 in term 1.
I20260809 07:49:20.592478 30224 raft_consensus.cc:2454] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6dcd603691e41d7b13a636e2fa66328 in term 1.
I20260809 07:49:20.592770 30011 leader_election.cc:302] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af6271f465e94800b3908f22c258a9f0, c6dcd603691e41d7b13a636e2fa66328; no voters:
I20260809 07:49:20.592933 30300 raft_consensus.cc:2781] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:49:20.593163 30300 raft_consensus.cc:686] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 1 LEADER]: Becoming Leader. State: Replica: c6dcd603691e41d7b13a636e2fa66328, State: Running, Role: LEADER
I20260809 07:49:20.593532 30300 consensus_queue.cc:237] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } }
I20260809 07:49:20.596716 29774 catalog_manager.cc:5495] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6dcd603691e41d7b13a636e2fa66328 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "c6dcd603691e41d7b13a636e2fa66328" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6dcd603691e41d7b13a636e2fa66328" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37323 } health_report { overall_health: HEALTHY } } }
W20260809 07:49:20.598660 29987 tablet.cc:2369] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:49:20.645923 30294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.642246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54012 (local address 127.18.71.66:37323)
0809 07:49:20.642388 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 07:49:20.642391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:49:20.642420 (+ 29us) server_negotiation.cc:407] Connection header received
0809 07:49:20.642501 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:49:20.642503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:49:20.642562 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:49:20.642645 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:49:20.642878 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.643371 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.644272 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:49:20.644565 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:49:20.644689 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:49:20.645032 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:49:20.645486 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:49:20.645622 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 07:49:20.645695 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 07:49:20.645753 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 07:49:20.651805 29935 raft_consensus.cc:1261] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 1 FOLLOWER]: Refusing update from remote peer c6dcd603691e41d7b13a636e2fa66328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:49:20.651804 30224 raft_consensus.cc:1261] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 1 FOLLOWER]: Refusing update from remote peer c6dcd603691e41d7b13a636e2fa66328: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:49:20.652576 30305 consensus_queue.cc:1035] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af6271f465e94800b3908f22c258a9f0" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 45263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:49:20.652791 30300 consensus_queue.cc:1035] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [LEADER]: Connected to new peer: Peer: permanent_uuid: "048c473f6d694ba381267c3b56e0f89e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:49:20.663228 30310 mvcc.cc:205] Tried to move back new op lower bound from 7316528171622338560 to 7316528171393748992. Current Snapshot: MvccSnapshot[applied={T|T < 7316528171622338560}]
I20260809 07:49:20.669459 29775 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:49:20.669638 29775 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:49:20.672138 29775 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6df9771d35d340338e325de221436f6a: Sending DeleteTablet for 3 replicas of tablet 2544e7a5cee147c1a6cfa2cf454df62e
I20260809 07:49:20.672945 30204 tablet_service.cc:1514] Processing DeleteTablet for tablet 2544e7a5cee147c1a6cfa2cf454df62e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:49:20 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:44802
I20260809 07:49:20.672945 29912 tablet_service.cc:1514] Processing DeleteTablet for tablet 2544e7a5cee147c1a6cfa2cf454df62e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:49:20 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40318
I20260809 07:49:20.673493 30317 tablet_replica.cc:330] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e: stopping tablet replica
I20260809 07:49:20.673636 30059 tablet_service.cc:1514] Processing DeleteTablet for tablet 2544e7a5cee147c1a6cfa2cf454df62e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:49:20 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:54000
I20260809 07:49:20.673961 30317 raft_consensus.cc:2229] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:49:20.674274 30318 tablet_replica.cc:330] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: stopping tablet replica
I20260809 07:49:20.674402 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29842
I20260809 07:49:20.674438 30317 raft_consensus.cc:2258] T 2544e7a5cee147c1a6cfa2cf454df62e P 048c473f6d694ba381267c3b56e0f89e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:49:20.674592 30318 raft_consensus.cc:2229] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:49:20.674969 30318 raft_consensus.cc:2258] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:49:20.679394 30319 tablet_replica.cc:330] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: stopping tablet replica
I20260809 07:49:20.679615 30319 raft_consensus.cc:2229] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:49:20.679888 30319 raft_consensus.cc:2258] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:49:20.681838 30319 ts_tablet_manager.cc:1905] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:49:20.682641 30318 ts_tablet_manager.cc:1905] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:49:20.684782 30319 ts_tablet_manager.cc:1918] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:49:20.684885 30319 log.cc:1196] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/2544e7a5cee147c1a6cfa2cf454df62e
I20260809 07:49:20.685276 30319 ts_tablet_manager.cc:1939] T 2544e7a5cee147c1a6cfa2cf454df62e P c6dcd603691e41d7b13a636e2fa66328: Deleting consensus metadata
I20260809 07:49:20.685463 30318 ts_tablet_manager.cc:1918] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:49:20.685571 30318 log.cc:1196] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/2544e7a5cee147c1a6cfa2cf454df62e
I20260809 07:49:20.685920 30318 ts_tablet_manager.cc:1939] T 2544e7a5cee147c1a6cfa2cf454df62e P af6271f465e94800b3908f22c258a9f0: Deleting consensus metadata
I20260809 07:49:20.686595 29759 catalog_manager.cc:4841] TS c6dcd603691e41d7b13a636e2fa66328 (127.18.71.66:37323): tablet 2544e7a5cee147c1a6cfa2cf454df62e (table test-table [id=a66cbdc268904e81aa982ad0666a5485]) successfully deleted
I20260809 07:49:20.687104 29761 catalog_manager.cc:4841] TS af6271f465e94800b3908f22c258a9f0 (127.18.71.67:45263): tablet 2544e7a5cee147c1a6cfa2cf454df62e (table test-table [id=a66cbdc268904e81aa982ad0666a5485]) successfully deleted
W20260809 07:49:20.692163 29760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:37041 (error 108)
I20260809 07:49:20.692939 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29991
I20260809 07:49:20.694064 30289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:49:20.692997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37041 (local address 127.0.0.1:40328)
0809 07:49:20.693917 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0809 07:49:20.693984 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:37041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":823}
W20260809 07:49:20.694767 29760 catalog_manager.cc:4584] TS 048c473f6d694ba381267c3b56e0f89e (127.18.71.65:37041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2544e7a5cee147c1a6cfa2cf454df62e: Network error: Client connection negotiation failed: client connection to 127.18.71.65:37041: connect: Connection refused (error 111)
I20260809 07:49:20.702267 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 30138
I20260809 07:49:20.710059 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 29741
2026-08-09T07:49:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2575 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:49:20 dist-test-slave-10pc krb5kdc[30323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:49:20 dist-test-slave-10pc krb5kdc[30323](info): set up 2 sockets
Aug 09 07:49:20 dist-test-slave-10pc krb5kdc[30323](info): commencing operation
krb5kdc: starting...
W20260809 07:49:21.796797 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:49:21 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:49:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/rangeradmin_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/rangeradmin_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/rangerlookup_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/rangerlookup_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
I20260809 07:49:21.976617 18717 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxTNb9d/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/postgres -l logfile start
2026-08-09 07:49:24.511 UTC [30354] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 07:49:24.511 UTC [30354] LOG: listening on IPv4 address "127.18.71.84", port 37159
2026-08-09 07:49:24.515 UTC [30354] LOG: listening on Unix socket "/tmp/.s.PGSQL.37159"
2026-08-09 07:49:24.537 UTC [30357] LOG: database system was shut down at 2026-08-09 07:49:23 UTC
2026-08-09 07:49:24.540 UTC [30354] LOG: database system is ready to accept connections
I20260809 07:49:25.548429 18717 mini_postgres.cc:96] Postgres bound to 37159
2026-08-09 07:49:25.556 UTC [30367] FATAL: database "slave" does not exist
127.18.71.84:37159 - accepting connections
I20260809 07:49:25.557511 18717 mini_ranger.cc:161] Starting Ranger...
I20260809 07:49:25.573377 18717 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 07:49:26.038516 18717 mini_ranger.cc:87] Created ranger Postgres database
I20260809 07:49:26.038600 18717 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-admin
2026-08-09 07:49:26,314 [I] DB FLAVOR :POSTGRES
2026-08-09 07:49:26,315 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 07:49:26,315 [I] Checking connection..
2026-08-09 07:49:26,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 07:49:26,578 [I] Checking connection passed.
2026-08-09 07:49:26,578 [I] --------- Verifying version history table ---------
2026-08-09 07:49:26,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 07:49:26,840 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 07:49:26,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 07:49:27,087 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 07:49:27,088 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 07:49:27,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 07:49:27.334 UTC [30438] WARNING: there is no transaction in progress
2026-08-09 07:49:27,341 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 07:49:27,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 07:49:27,604 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 07:49:27,604 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 07:49:27,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 07:49:27,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 07:49:28,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-10pc.c.gcp-upstream.internal','N') ;"
2026-08-09 07:49:28,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 07:49:28,575 [I] Table x_portal_user does not exist in database ranger
2026-08-09 07:49:28,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 07:49:28,834 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 07:49:28,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 07:49:29,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 07:49:29,343 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 07:49:29,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 07:49:30.298 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.314 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.331 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.348 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.365 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.384 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.440 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.447 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.461 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.471 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.485 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.498 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.508 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.514 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.521 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:30.528 UTC [30554] WARNING: there is no transaction in progress
2026-08-09 07:49:31,150 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 07:49:31,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 07:49:31,404 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 07:49:31,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 07:49:31,680 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 07:49:31,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 07:49:31,944 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 07:49:31,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 07:49:32,198 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 07:49:32,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-10pc.c.gcp-upstream.internal';"
2026-08-09 07:49:32,703 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 07:49:32,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 07:49:32,956 [I] DB_PATCHES have already been applied
I20260809 07:49:32.962123 18717 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxTNb9d/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 07:49:32.962193 18717 mini_ranger.cc:193] Using host: 127.18.71.84
I20260809 07:49:32.966897 18717 mini_ranger.cc:227] Ranger admin URL: http://127.18.71.84:49343
Aug 09, 2026 7:49:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:49:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:49:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:49:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:49:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:49:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:49:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 07:49:34.114135 18717 mini_ranger.cc:160] Time spent starting Ranger: real 8.557s user 0.002s sys 0.010s
Aug 09, 2026 7:49:54 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:49:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 7:49:54 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:49:54 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:49:55 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 07:49:56.059525 18717 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/rangerkms_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/rangerkms_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.71.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.71.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 07:49:56 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261796, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 07:49:56.152882 18717 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 07:49:56.163103 18717 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 07:49:57.015854 18717 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 07:49:57.016000 18717 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms
I20260809 07:49:57.016140 18717 mini_ranger_kms.cc:227] Using postgres at 127.18.71.84:37159
2026-08-09 07:49:57,221 [I] DB FLAVOR :POSTGRES
2026-08-09 07:49:57,221 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 07:49:57,222 [I] Checking connection
2026-08-09 07:49:57,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 07:49:57,477 [I] connection success
2026-08-09 07:49:57,477 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 07:49:57,477 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 07:49:57,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 07:49:57,787 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 07:49:57,788 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 07:49:57,788 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 07:49:57,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:37159/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 07:49:58,065 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 07:49:58.518544 18717 mini_ranger_kms.cc:310] Created kms service
I20260809 07:49:58.741925 18717 mini_ranger_kms.cc:326] Created kudu user
I20260809 07:49:58.786127 18717 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 07:49:58 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261796, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:49:59.161875 18717 mini_ranger_kms.cc:384] Added ranger policy
I20260809 07:49:59.162261 18717 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 07:49:59.162293 18717 mini_ranger_kms.cc:240] Using host: 127.18.71.84
I20260809 07:49:59.167055 18717 mini_ranger_kms.cc:276] Ranger KMS PID: 30754
I20260809 07:49:59.167199 18717 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.71.84:55567
Aug 09, 2026 7:50:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:50:00 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:50:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:50:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:50:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:50:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 07:50:01.639585 18717 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.487s user 0.016s sys 0.174s
I20260809 07:50:01.639760 18717 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 07:50:01.639909 18717 mini_ranger_kms.cc:401] 127.18.71.84:55567/kms/v1/keys
Aug 09 07:50:01 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261801, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.71.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:50:02 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261801, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.71.84@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
Aug 09 07:50:02 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261801, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.71.84@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
Aug 09 07:50:04 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:50:04.832535 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:36139
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:36139
--ranger_config_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:55567/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:50:04.971923 30813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:04.972262 30813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:04.972342 30813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:04.976547 30813 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:04.976639 30813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:04.976675 30813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:04.976713 30813 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:04.976748 30813 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:04.982538 30813 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.71.84:55567/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:36139
--ranger_config_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:36139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:04.983789 30813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:04.985010 30813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:04.995453 30822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:04.995755 30813 server_base.cc:1034] running on GCE node
W20260809 07:50:04.995464 30818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:04.995464 30820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:04.996547 30813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:04.997781 30813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:04.998992 30813 hybrid_clock.cc:648] HybridClock initialized: now 1786261804998951 us; error 61 us; skew 500 ppm
Aug 09 07:50:05 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261805, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:05.006063 30813 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:50:05.007335 30813 webserver.cc:460] Webserver started at http://127.18.71.126:35035/ using document root <none> and password file <none>
I20260809 07:50:05.008033 30813 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:05.008121 30813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:05.008355 30813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:05 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261805, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:05.057761 30813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "c4d54448658d472c844e4006d2147d62"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-10pc"
server_key: "5853718ff4316d2b3d1b52e5fe0205de"
server_key_iv: "2fe7a41023774c96412b64510f16f060"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:05.058552 30813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "c4d54448658d472c844e4006d2147d62"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-10pc"
server_key: "5853718ff4316d2b3d1b52e5fe0205de"
server_key_iv: "2fe7a41023774c96412b64510f16f060"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:05.063714 30813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 07:50:05.081342 30829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:05.083158 30813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:05.083314 30813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "c4d54448658d472c844e4006d2147d62"
format_stamp: "Formatted at 2026-08-09 07:50:05 on dist-test-slave-10pc"
server_key: "5853718ff4316d2b3d1b52e5fe0205de"
server_key_iv: "2fe7a41023774c96412b64510f16f060"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:05.083464 30813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:05.102241 30813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:05.105695 30813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:05.105901 30813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:05.114454 30813 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:36139
I20260809 07:50:05.114490 30882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:36139 every 8 connection(s)
I20260809 07:50:05.115660 30813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:05.118731 30884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:05.122875 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 30813
I20260809 07:50:05.123009 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:50:05.123337 18717 external_mini_cluster.cc:1428] Setting key 72795ba5de1b4701173178cfd4282ff4
I20260809 07:50:05.125702 30884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62: Bootstrap starting.
I20260809 07:50:05.128276 30884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:05.129145 30884 log.cc:824] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:05.131503 30884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62: No bootstrap required, opened a new log
Aug 09 07:50:05 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:05.136262 30884 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d54448658d472c844e4006d2147d62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36139 } }
I20260809 07:50:05.136583 30884 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:05.136672 30884 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d54448658d472c844e4006d2147d62, State: Initialized, Role: FOLLOWER
I20260809 07:50:05.137213 30884 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d54448658d472c844e4006d2147d62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36139 } }
I20260809 07:50:05.137475 30884 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:50:05.137553 30884 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:50:05.137672 30884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:05.138839 30884 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d54448658d472c844e4006d2147d62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36139 } }
I20260809 07:50:05.139212 30887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:05.124856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36994 (local address 127.18.71.126:36139)
0809 07:50:05.125369 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:05.125389 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:05.125435 (+ 46us) server_negotiation.cc:407] Connection header received
0809 07:50:05.126293 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:05.126317 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:05.126746 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:05.127125 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:05.127986 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:05.129541 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:05.130573 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:05.131064 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:05.131268 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:05.133941 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:05.133946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:05.133961 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:05.133987 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:05.135738 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:05.136214 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:05.136224 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:05.136229 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:05.136336 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:05.136693 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:05.136696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:05.136698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:05.137213 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:05.137434 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:05.138122 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 07:50:05.138369 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":149,"threads_started":1}
I20260809 07:50:05.139240 30884 leader_election.cc:302] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4d54448658d472c844e4006d2147d62; no voters:
I20260809 07:50:05.139591 30884 leader_election.cc:288] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:50:05.139712 30890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:50:05.139971 30890 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [term 1 LEADER]: Becoming Leader. State: Replica: c4d54448658d472c844e4006d2147d62, State: Running, Role: LEADER
I20260809 07:50:05.140298 30890 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d54448658d472c844e4006d2147d62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36139 } }
I20260809 07:50:05.141000 30884 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:05.141649 30890 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4d54448658d472c844e4006d2147d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d54448658d472c844e4006d2147d62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36139 } } }
I20260809 07:50:05.141804 30890 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:05.142037 30890 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4d54448658d472c844e4006d2147d62. Latest consensus state: current_term: 1 leader_uuid: "c4d54448658d472c844e4006d2147d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d54448658d472c844e4006d2147d62" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36139 } } }
I20260809 07:50:05.142143 30890 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:05.143477 30884 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 07:50:05.144713 30897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:50:05.151635 30897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:50:05.162664 30897 catalog_manager.cc:1347] Generated new cluster ID: 2fec03cf81c8423992fa62873fd76c47
I20260809 07:50:05.162740 30897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:50:05.182205 30897 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:50:05.183662 30897 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:50:05.194140 30897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62: Generated new TSK 0
I20260809 07:50:05.195062 30897 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:50:06.000279 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:55567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36139
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:50:06.152220 30933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:06.152529 30933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:06.152614 30933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:06.156878 30933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:06.156963 30933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:06.157051 30933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:50:06.163245 30933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.71.84:55567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:06.164394 30933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:06.165483 30933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:06.175482 30940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.175343 30942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.175343 30944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:06.176005 30933 server_base.cc:1034] running on GCE node
I20260809 07:50:06.176445 30933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:06.177201 30933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:06.178380 30933 hybrid_clock.cc:648] HybridClock initialized: now 1786261806178360 us; error 40 us; skew 500 ppm
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:06.181663 30933 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:50:06.182653 30933 webserver.cc:460] Webserver started at http://127.18.71.65:34345/ using document root <none> and password file <none>
I20260809 07:50:06.183251 30933 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:06.183342 30933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:06.183588 30933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:06.199919 30933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "ca98381c68c3400f8b5013c47723b76a"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "7fa3aa11fda17d2fc120d17477c8abf5"
server_key_iv: "6b9490d927b109683dfabd32805e9e0b"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:06.200655 30933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "ca98381c68c3400f8b5013c47723b76a"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "7fa3aa11fda17d2fc120d17477c8abf5"
server_key_iv: "6b9490d927b109683dfabd32805e9e0b"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:06.209622 30933 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260809 07:50:06.233297 30953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.235260 30933 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260809 07:50:06.235472 30933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "ca98381c68c3400f8b5013c47723b76a"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "7fa3aa11fda17d2fc120d17477c8abf5"
server_key_iv: "6b9490d927b109683dfabd32805e9e0b"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:06.235791 30933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:06.252969 30933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:06.256935 30933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:06.257205 30933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:06.258054 30933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:06.259259 30933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:06.259373 30933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.259503 30933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:06.259590 30933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.273434 30933 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:34247
I20260809 07:50:06.274619 30933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:06.278708 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 30933
I20260809 07:50:06.278826 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:50:06.279161 18717 external_mini_cluster.cc:1428] Setting key 5589803bd78b5705eb0afb5e5de281df
I20260809 07:50:06.275908 31077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:34247 every 8 connection(s)
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:50:06.293155 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:06.278517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:59507 (local address 127.18.71.126:36139)
0809 07:50:06.280304 (+ 1787us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:06.280309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:06.280322 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:50:06.280361 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:06.280364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:06.280409 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:06.282283 (+ 1874us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:06.282392 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.283255 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.284989 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.285351 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.285436 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:06.288553 (+ 3117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:06.288557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:06.288560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:06.288585 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:06.290484 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:06.291122 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:06.291126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:06.291128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:06.291225 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:06.291656 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:06.291660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:06.291662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:06.291886 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:06.291957 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:06.292808 (+ 851us) server_negotiation.cc:299] Negotiation successful
0809 07:50:06.292954 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1711,"thread_start_us":75,"threads_started":1}
I20260809 07:50:06.294931 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:06.277294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36139 (local address 127.18.71.65:59507)
0809 07:50:06.277936 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:06.278022 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:06.279371 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:06.281065 (+ 1694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:06.281075 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:06.281438 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:06.282131 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:06.282153 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.283405 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.283409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:06.284822 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:06.284833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.285664 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.285668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:06.285867 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:06.286580 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:06.286609 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:06.288323 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:06.290628 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:06.290633 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:06.290649 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:06.290998 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:06.291350 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:06.291353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:06.291356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:06.291527 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:06.292043 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:06.292050 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:06.292463 (+ 413us) client_negotiation.cc:769] Sending connection context
0809 07:50:06.293385 (+ 922us) client_negotiation.cc:240] Negotiation successful
0809 07:50:06.293772 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":170,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
I20260809 07:50:06.296483 31078 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36139
I20260809 07:50:06.296757 31078 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:06.297370 31078 heartbeater.cc:511] Master 127.18.71.126:36139 requested a full tablet report, sending...
I20260809 07:50:06.299125 30847 ts_manager.cc:194] Registered new tserver with Master: ca98381c68c3400f8b5013c47723b76a (127.18.71.65:34247)
I20260809 07:50:06.300693 30847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:59507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:50:06.316324 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:55567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36139
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 07:50:06.478871 31084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:06.479368 31084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:06.479527 31084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:06.486500 31084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:06.486681 31084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:06.486843 31084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:50:06.495759 31084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.71.84:55567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:06.497328 31084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:06.498862 31084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:06.512672 31093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.512676 31091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.513356 31090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:06.513995 31084 server_base.cc:1034] running on GCE node
I20260809 07:50:06.514534 31084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:06.515429 31084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:06.516628 31084 hybrid_clock.cc:648] HybridClock initialized: now 1786261806516588 us; error 54 us; skew 500 ppm
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:06.520550 31084 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:50:06.521772 31084 webserver.cc:460] Webserver started at http://127.18.71.66:33535/ using document root <none> and password file <none>
I20260809 07:50:06.522624 31084 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:06.522732 31084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:06.523000 31084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:06.543220 31084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "aae4eaa7d6b2464ab6a02b6d4cece844"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "18b088069854c0feae31dbed1d0f3b62"
server_key_iv: "a0c97bb9827e277ea4fb5f3d73dab69e"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:06.543995 31084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "aae4eaa7d6b2464ab6a02b6d4cece844"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "18b088069854c0feae31dbed1d0f3b62"
server_key_iv: "a0c97bb9827e277ea4fb5f3d73dab69e"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:06.549770 31084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 07:50:06.580117 31101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.582156 31084 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s
I20260809 07:50:06.582317 31084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "aae4eaa7d6b2464ab6a02b6d4cece844"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "18b088069854c0feae31dbed1d0f3b62"
server_key_iv: "a0c97bb9827e277ea4fb5f3d73dab69e"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:06.582491 31084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:06.619714 31084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:06.623638 31084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:06.623911 31084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:06.624784 31084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:06.625937 31084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:06.626049 31084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.626150 31084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:06.626216 31084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:06.640275 31084 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:35759
I20260809 07:50:06.641613 31084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:50:06.643111 31227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:35759 every 8 connection(s)
I20260809 07:50:06.647976 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 31084
I20260809 07:50:06.648097 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:50:06.648429 18717 external_mini_cluster.cc:1428] Setting key 329aa22cb27eead4841bf1c737251148
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:06.679827 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:06.646860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:53125 (local address 127.18.71.126:36139)
0809 07:50:06.647021 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:06.647027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:06.657165 (+ 10138us) server_negotiation.cc:407] Connection header received
0809 07:50:06.659715 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:06.659719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:06.659782 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:06.659845 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:06.663147 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.664032 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.670001 (+ 5969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.670374 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.670464 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:06.675109 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:06.675114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:06.675117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:06.675157 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:06.677060 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:06.677793 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:06.677797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:06.677799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:06.677892 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:06.678360 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:06.678364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:06.678366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:06.678610 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:06.678697 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:06.679507 (+ 810us) server_negotiation.cc:299] Negotiation successful
0809 07:50:06.679653 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 07:50:06.682201 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:06.645747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36139 (local address 127.18.71.66:53125)
0809 07:50:06.656921 (+ 11174us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:06.657028 (+ 107us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:06.658393 (+ 1365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:06.660377 (+ 1984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:06.660388 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:06.660809 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:06.661569 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:06.661591 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.668137 (+ 6546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.668140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:06.669818 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:06.669831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:06.670743 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:06.670747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:06.670969 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:06.672947 (+ 1978us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:06.672989 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:06.674824 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:06.677216 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:06.677224 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:06.677246 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:06.677642 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:06.678015 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:06.678018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:06.678021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:06.678228 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:06.678771 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:06.678777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:06.679182 (+ 405us) client_negotiation.cc:769] Sending connection context
0809 07:50:06.680382 (+ 1200us) client_negotiation.cc:240] Negotiation successful
0809 07:50:06.680938 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10828,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:50:06.686738 31228 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36139
I20260809 07:50:06.687031 31228 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:06.687711 31228 heartbeater.cc:511] Master 127.18.71.126:36139 requested a full tablet report, sending...
I20260809 07:50:06.689018 30847 ts_manager.cc:194] Registered new tserver with Master: aae4eaa7d6b2464ab6a02b6d4cece844 (127.18.71.66:35759)
I20260809 07:50:06.689805 30847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:53125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:50:06.709256 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:55567/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36139
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
W20260809 07:50:06.920809 31233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:06.921579 31233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:06.921712 31233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:06.927668 31233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:06.927845 31233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:06.927991 31233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:50:06.935300 31233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.71.84:55567/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:06.937213 31233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:06.938645 31233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:06.951786 31241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.952337 31240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:06.952523 31243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:06.953509 31233 server_base.cc:1034] running on GCE node
I20260809 07:50:06.954147 31233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:06.954914 31233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:06.970358 31233 hybrid_clock.cc:648] HybridClock initialized: now 1786261806970246 us; error 131 us; skew 500 ppm
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:06.974396 31233 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:50:06.975659 31233 webserver.cc:460] Webserver started at http://127.18.71.67:34813/ using document root <none> and password file <none>
I20260809 07:50:06.976501 31233 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:06.976651 31233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:06.976939 31233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:06 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:06.992936 31233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "d1310654c33b4835a3cb10ff9ec670e2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "8982f2afbf46da6d8116fee2161aee30"
server_key_iv: "ee3d70a68a7d1a01511eca1933c57d32"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:06.993811 31233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "d1310654c33b4835a3cb10ff9ec670e2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "8982f2afbf46da6d8116fee2161aee30"
server_key_iv: "ee3d70a68a7d1a01511eca1933c57d32"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:07.000087 31233 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260809 07:50:07.021656 31250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.023483 31233 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260809 07:50:07.023713 31233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "d1310654c33b4835a3cb10ff9ec670e2"
format_stamp: "Formatted at 2026-08-09 07:50:06 on dist-test-slave-10pc"
server_key: "8982f2afbf46da6d8116fee2161aee30"
server_key_iv: "ee3d70a68a7d1a01511eca1933c57d32"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:07.023942 31233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:07.051791 31233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:07.055510 31233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:07.055771 31233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:07.056478 31233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:07.057744 31233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:07.057874 31233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.057996 31233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:07.058074 31233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:07.068615 31233 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:35241
I20260809 07:50:07.070129 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 31233
I20260809 07:50:07.070261 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:50:07.070566 18717 external_mini_cluster.cc:1428] Setting key a3a8d885956cf047ab3cd4c83c30c41a
I20260809 07:50:07.069866 31233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:50:07.075835 31376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:35241 every 8 connection(s)
Aug 09 07:50:07 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:07.092840 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.079112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:43691 (local address 127.18.71.126:36139)
0809 07:50:07.079225 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.079230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.079245 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:50:07.079911 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.079915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.079965 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.080015 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:07.081741 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.082556 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.084877 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.085243 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.085318 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.088602 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:07.088607 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:07.088609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:07.088637 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:07.090434 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:07.091057 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:07.091060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:07.091062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:07.091147 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:07.091535 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:07.091538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:07.091540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:07.091755 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:07.091833 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.092538 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.092670 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:50:07.095252 31379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.078060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36139 (local address 127.18.71.67:43691)
0809 07:50:07.078576 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.078630 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.079662 (+ 1032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.080433 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.080443 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.080839 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:07.081565 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.081585 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.082720 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.082724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.084742 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.084753 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.085536 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.085539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.085752 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.086480 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:07.086502 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:07.088355 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:07.090586 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:07.090592 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:07.090611 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:07.090941 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:07.091247 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:07.091250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:07.091252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:07.091420 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:07.091901 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:07.091908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:07.092229 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.093529 (+ 1300us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.093877 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":154,"threads_started":1}
I20260809 07:50:07.097003 31377 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36139
I20260809 07:50:07.097285 31377 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:07.097888 31377 heartbeater.cc:511] Master 127.18.71.126:36139 requested a full tablet report, sending...
I20260809 07:50:07.099138 30847 ts_manager.cc:194] Registered new tserver with Master: d1310654c33b4835a3cb10ff9ec670e2 (127.18.71.67:35241)
I20260809 07:50:07.099869 30847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:43691
I20260809 07:50:07.108497 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:50:07.124368 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.113724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34986 (local address 127.18.71.126:36139)
0809 07:50:07.113843 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.113848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.114578 (+ 730us) server_negotiation.cc:407] Connection header received
0809 07:50:07.114781 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.114784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.114833 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.114883 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:07.115925 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.116810 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.118084 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.118433 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.118504 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.120548 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:07.120552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:07.120555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:07.120584 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:07.122451 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:07.122994 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:07.122998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:07.123000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:07.123092 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:07.123419 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:07.123422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:07.123424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:07.123649 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:07.123779 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.124085 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.124218 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:50:07.127945 30847 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:50:07.132385 30847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:50:07.157363 31390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.143316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:34247 (local address 127.0.0.1:33036)
0809 07:50:07.148978 (+ 5662us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.149040 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.149195 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.150667 (+ 1472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.150670 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.150690 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:07.150811 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.150817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.153440 (+ 2623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.153444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.155389 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.155400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.156784 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.156787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.156916 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.156954 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.157071 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.157196 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5467,"mutex_wait_us":7,"spinlock_wait_cycles":768,"thread_start_us":116,"threads_started":1}
I20260809 07:50:07.157405 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.144209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33036 (local address 127.18.71.65:34247)
0809 07:50:07.145476 (+ 1267us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.145484 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.149139 (+ 3655us) server_negotiation.cc:407] Connection header received
0809 07:50:07.149361 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.149367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.149614 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.149784 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:07.150926 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.152083 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.155553 (+ 3470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.156404 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.156513 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.156681 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.157141 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.157254 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1152,"thread_start_us":92,"threads_started":1}
I20260809 07:50:07.161559 31006 tablet_service.cc:1467] Processing CreateTablet for tablet cf28b2e69b9a4462b376529378e18c2c (DEFAULT_TABLE table=test-table [id=908292d81c614aa8a3c62d0ebe5251a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:50:07.165005 31006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf28b2e69b9a4462b376529378e18c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:07.166414 31394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.145826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:35241 (local address 127.0.0.1:60080)
0809 07:50:07.149701 (+ 3875us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.149712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.149790 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.150490 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.150496 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.150532 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:07.150795 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.150802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.159543 (+ 8741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.159546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.160797 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.160807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.166001 (+ 5194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.166005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.166092 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.166116 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.166186 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.166265 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3819,"mutex_wait_us":119,"thread_start_us":92,"threads_started":1}
I20260809 07:50:07.166877 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.146003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60080 (local address 127.18.71.67:35241)
0809 07:50:07.146468 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.146480 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.149887 (+ 3407us) server_negotiation.cc:407] Connection header received
0809 07:50:07.149972 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.149977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.150175 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.150327 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:07.154122 (+ 3795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.159379 (+ 5257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.164852 (+ 5473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.165683 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.165800 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.165934 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.166626 (+ 692us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.166741 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":173,"threads_started":1}
I20260809 07:50:07.170672 31293 tablet_service.cc:1467] Processing CreateTablet for tablet cf28b2e69b9a4462b376529378e18c2c (DEFAULT_TABLE table=test-table [id=908292d81c614aa8a3c62d0ebe5251a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:50:07.171830 31293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf28b2e69b9a4462b376529378e18c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:07.173139 31393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.145391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.18.71.66:35759)
0809 07:50:07.145685 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.145703 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.149945 (+ 4242us) server_negotiation.cc:407] Connection header received
0809 07:50:07.150044 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.150049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.150256 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.150423 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:07.153641 (+ 3218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.154752 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.163055 (+ 8303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.163925 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.164055 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.164202 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.172869 (+ 8667us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.172994 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":97,"threads_started":1}
I20260809 07:50:07.173588 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.145078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35759 (local address 127.0.0.1:44146)
0809 07:50:07.149428 (+ 4350us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.149440 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.149529 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.150581 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.150584 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.150604 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:07.152902 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.152914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.154910 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.154914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.158149 (+ 3235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.158160 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.172552 (+ 14392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.172556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.172669 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.172695 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.173338 (+ 643us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.173441 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4291,"mutex_wait_us":2124,"spinlock_wait_cycles":3328,"thread_start_us":100,"threads_started":1}
Aug 09 07:50:07 dist-test-slave-10pc krb5kdc[30323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261806, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:07.177183 31149 tablet_service.cc:1467] Processing CreateTablet for tablet cf28b2e69b9a4462b376529378e18c2c (DEFAULT_TABLE table=test-table [id=908292d81c614aa8a3c62d0ebe5251a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:50:07.178340 31149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf28b2e69b9a4462b376529378e18c2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:07.180752 31398 tablet_bootstrap.cc:492] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Bootstrap starting.
I20260809 07:50:07.183851 31398 tablet_bootstrap.cc:654] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:07.184837 31398 log.cc:824] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:07.188213 31400 tablet_bootstrap.cc:492] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a: Bootstrap starting.
I20260809 07:50:07.190841 31400 tablet_bootstrap.cc:654] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:07.191772 31400 log.cc:824] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:07.194000 31400 tablet_bootstrap.cc:492] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a: No bootstrap required, opened a new log
I20260809 07:50:07.194202 31400 ts_tablet_manager.cc:1397] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:50:07.194818 31401 tablet_bootstrap.cc:492] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844: Bootstrap starting.
I20260809 07:50:07.196991 31401 tablet_bootstrap.cc:654] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:07.198429 31401 log.cc:824] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:07.198865 31398 tablet_bootstrap.cc:492] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: No bootstrap required, opened a new log
I20260809 07:50:07.199084 31398 ts_tablet_manager.cc:1397] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.000s
I20260809 07:50:07.200357 31401 tablet_bootstrap.cc:492] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844: No bootstrap required, opened a new log
I20260809 07:50:07.200290 31400 raft_consensus.cc:348] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.200567 31401 ts_tablet_manager.cc:1397] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260809 07:50:07.200606 31400 raft_consensus.cc:374] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:07.200676 31400 raft_consensus.cc:729] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca98381c68c3400f8b5013c47723b76a, State: Initialized, Role: FOLLOWER
I20260809 07:50:07.201161 31400 consensus_queue.cc:260] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.202355 31400 ts_tablet_manager.cc:1428] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260809 07:50:07.202798 31078 heartbeater.cc:503] Master 127.18.71.126:36139 was elected leader, sending a full tablet report...
I20260809 07:50:07.203524 31398 raft_consensus.cc:348] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.203785 31398 raft_consensus.cc:374] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:07.203838 31398 raft_consensus.cc:729] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1310654c33b4835a3cb10ff9ec670e2, State: Initialized, Role: FOLLOWER
I20260809 07:50:07.204406 31398 consensus_queue.cc:260] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.205739 31398 ts_tablet_manager.cc:1428] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260809 07:50:07.206213 31377 heartbeater.cc:503] Master 127.18.71.126:36139 was elected leader, sending a full tablet report...
I20260809 07:50:07.209724 31401 raft_consensus.cc:348] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.210176 31401 raft_consensus.cc:374] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:07.210304 31401 raft_consensus.cc:729] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae4eaa7d6b2464ab6a02b6d4cece844, State: Initialized, Role: FOLLOWER
I20260809 07:50:07.210980 31401 consensus_queue.cc:260] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.211854 31228 heartbeater.cc:503] Master 127.18.71.126:36139 was elected leader, sending a full tablet report...
I20260809 07:50:07.213608 31401 ts_tablet_manager.cc:1428] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
W20260809 07:50:07.280616 31079 tablet.cc:2369] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:50:07.290606 31404 raft_consensus.cc:482] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:50:07.290833 31404 raft_consensus.cc:504] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.292236 31404 leader_election.cc:288] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aae4eaa7d6b2464ab6a02b6d4cece844 (127.18.71.66:35759), d1310654c33b4835a3cb10ff9ec670e2 (127.18.71.67:35241)
I20260809 07:50:07.303285 31406 raft_consensus.cc:482] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:50:07.303493 31406 raft_consensus.cc:504] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.304821 31406 leader_election.cc:288] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca98381c68c3400f8b5013c47723b76a (127.18.71.65:34247), d1310654c33b4835a3cb10ff9ec670e2 (127.18.71.67:35241)
I20260809 07:50:07.307823 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.293856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:36343 (local address 127.18.71.67:35241)
0809 07:50:07.293980 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.293985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.302508 (+ 8523us) server_negotiation.cc:407] Connection header received
0809 07:50:07.302727 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.302730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.302786 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.302829 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:07.303232 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.303924 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.306513 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.307133 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.307228 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.307289 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.307623 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.307699 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:50:07.308420 31408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.293185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:35241 (local address 127.18.71.65:36343)
0809 07:50:07.302399 (+ 9214us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.302414 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.302587 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.302944 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.302945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.302958 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:07.303091 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.303098 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.304112 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.304117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.306371 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.306384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.307390 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.307393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.307485 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.307508 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.308187 (+ 679us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.308266 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9143,"thread_start_us":136,"threads_started":1}
I20260809 07:50:07.309219 31325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf28b2e69b9a4462b376529378e18c2c" candidate_uuid: "ca98381c68c3400f8b5013c47723b76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1310654c33b4835a3cb10ff9ec670e2" is_pre_election: true
I20260809 07:50:07.309581 31325 raft_consensus.cc:2454] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca98381c68c3400f8b5013c47723b76a in term 0.
I20260809 07:50:07.310159 31393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.293695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:41369 (local address 127.18.71.66:35759)
0809 07:50:07.293841 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.293846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.299360 (+ 5514us) server_negotiation.cc:407] Connection header received
0809 07:50:07.299511 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.299514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.299571 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.299666 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:07.300062 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.300865 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.308810 (+ 7945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.309433 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.309525 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.309570 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.309955 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.310048 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:50:07.310221 30957 leader_election.cc:302] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca98381c68c3400f8b5013c47723b76a, d1310654c33b4835a3cb10ff9ec670e2; no voters:
I20260809 07:50:07.310541 31404 raft_consensus.cc:2781] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:50:07.310632 31404 raft_consensus.cc:482] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:50:07.310701 31404 raft_consensus.cc:3037] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:07.310755 31409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.292727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35759 (local address 127.18.71.65:41369)
0809 07:50:07.299247 (+ 6520us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.299264 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.299404 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.299789 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.299793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.299813 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:07.299913 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.299919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.301032 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.301035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.302350 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.302362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.309667 (+ 7305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.309671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.309765 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.309838 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.310504 (+ 666us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.310583 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6437,"thread_start_us":129,"threads_started":1}
I20260809 07:50:07.311291 31155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf28b2e69b9a4462b376529378e18c2c" candidate_uuid: "ca98381c68c3400f8b5013c47723b76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" is_pre_election: true
I20260809 07:50:07.311491 31155 raft_consensus.cc:2454] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca98381c68c3400f8b5013c47723b76a in term 0.
I20260809 07:50:07.312086 31404 raft_consensus.cc:504] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.312754 31155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf28b2e69b9a4462b376529378e18c2c" candidate_uuid: "ca98381c68c3400f8b5013c47723b76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844"
I20260809 07:50:07.312865 31155 raft_consensus.cc:3037] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:07.313243 31325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf28b2e69b9a4462b376529378e18c2c" candidate_uuid: "ca98381c68c3400f8b5013c47723b76a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1310654c33b4835a3cb10ff9ec670e2"
I20260809 07:50:07.313364 31325 raft_consensus.cc:3037] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:07.314131 31155 raft_consensus.cc:2454] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca98381c68c3400f8b5013c47723b76a in term 1.
I20260809 07:50:07.314698 31325 raft_consensus.cc:2454] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca98381c68c3400f8b5013c47723b76a in term 1.
I20260809 07:50:07.312466 31404 leader_election.cc:288] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [CANDIDATE]: Term 1 election: Requested vote from peers aae4eaa7d6b2464ab6a02b6d4cece844 (127.18.71.66:35759), d1310654c33b4835a3cb10ff9ec670e2 (127.18.71.67:35241)
I20260809 07:50:07.314459 30956 leader_election.cc:302] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aae4eaa7d6b2464ab6a02b6d4cece844, ca98381c68c3400f8b5013c47723b76a; no voters:
I20260809 07:50:07.315656 31404 raft_consensus.cc:2781] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:50:07.315876 31404 raft_consensus.cc:686] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [term 1 LEADER]: Becoming Leader. State: Replica: ca98381c68c3400f8b5013c47723b76a, State: Running, Role: LEADER
I20260809 07:50:07.316277 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.306197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:59023 (local address 127.18.71.67:35241)
0809 07:50:07.307998 (+ 1801us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.308003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.308884 (+ 881us) server_negotiation.cc:407] Connection header received
0809 07:50:07.309070 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.309073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.309114 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.309157 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:07.309690 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.310323 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.314960 (+ 4637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.315534 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.315627 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.315679 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.316088 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.316159 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1729,"thread_start_us":97,"threads_started":1}
I20260809 07:50:07.316226 31404 consensus_queue.cc:237] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } }
I20260809 07:50:07.316661 31412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.305588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:35241 (local address 127.18.71.66:59023)
0809 07:50:07.308811 (+ 3223us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.308824 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.308947 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.309292 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.309295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.309310 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:07.309559 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.309567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.310459 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.310462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.311637 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.311651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.315779 (+ 4128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.315783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.315909 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.315935 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.316386 (+ 451us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.316484 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3154,"thread_start_us":94,"threads_started":1}
I20260809 07:50:07.317137 31325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf28b2e69b9a4462b376529378e18c2c" candidate_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1310654c33b4835a3cb10ff9ec670e2" is_pre_election: true
I20260809 07:50:07.317312 31325 raft_consensus.cc:2379] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aae4eaa7d6b2464ab6a02b6d4cece844 in current term 1: Already voted for candidate ca98381c68c3400f8b5013c47723b76a in this term.
I20260809 07:50:07.321182 30846 catalog_manager.cc:5495] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a reported cstate change: term changed from 0 to 1, leader changed from <none> to ca98381c68c3400f8b5013c47723b76a (127.18.71.65). New cstate: current_term: 1 leader_uuid: "ca98381c68c3400f8b5013c47723b76a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca98381c68c3400f8b5013c47723b76a" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 34247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 } health_report { overall_health: UNKNOWN } } }
I20260809 07:50:07.323249 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.306070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:55541 (local address 127.18.71.65:34247)
0809 07:50:07.306187 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.306191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.306609 (+ 418us) server_negotiation.cc:407] Connection header received
0809 07:50:07.306769 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.306772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.306807 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.306846 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:07.307223 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.307893 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.321045 (+ 13152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.322379 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.322477 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.322534 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.323059 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.323140 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 07:50:07.323771 31411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.305191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:34247 (local address 127.18.71.66:55541)
0809 07:50:07.306531 (+ 1340us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.306545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.306670 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.306950 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:07.306953 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:07.306967 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:07.307100 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.307106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.308044 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.308358 (+ 314us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.320890 (+ 12532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:07.320903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.322740 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.322744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:07.322909 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.322933 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:50:07.323555 (+ 622us) client_negotiation.cc:240] Negotiation successful
0809 07:50:07.323634 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1254,"thread_start_us":111,"threads_started":1}
I20260809 07:50:07.324625 31016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf28b2e69b9a4462b376529378e18c2c" candidate_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca98381c68c3400f8b5013c47723b76a" is_pre_election: true
I20260809 07:50:07.325392 31105 leader_election.cc:302] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aae4eaa7d6b2464ab6a02b6d4cece844; no voters: ca98381c68c3400f8b5013c47723b76a, d1310654c33b4835a3cb10ff9ec670e2
I20260809 07:50:07.325857 31406 raft_consensus.cc:2726] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 07:50:07.327728 31378 tablet.cc:2369] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:50:07.354957 31391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.349588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33038 (local address 127.18.71.65:34247)
0809 07:50:07.349710 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:07.349715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:07.349728 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:50:07.349814 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:07.349817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:07.349886 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:07.349992 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:50:07.350378 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.351188 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.352492 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:07.352833 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:07.352936 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:07.353353 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:50:07.353951 (+ 598us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:50:07.354072 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:07.354729 (+ 657us) server_negotiation.cc:299] Negotiation successful
0809 07:50:07.354787 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:50:07.366930 31325 raft_consensus.cc:1261] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 1 FOLLOWER]: Refusing update from remote peer ca98381c68c3400f8b5013c47723b76a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:07.367122 31155 raft_consensus.cc:1261] T cf28b2e69b9a4462b376529378e18c2c P aae4eaa7d6b2464ab6a02b6d4cece844 [term 1 FOLLOWER]: Refusing update from remote peer ca98381c68c3400f8b5013c47723b76a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:07.367810 31404 consensus_queue.cc:1035] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae4eaa7d6b2464ab6a02b6d4cece844" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:07.368597 31404 consensus_queue.cc:1035] T cf28b2e69b9a4462b376529378e18c2c P ca98381c68c3400f8b5013c47723b76a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1310654c33b4835a3cb10ff9ec670e2" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:07.386567 31418 mvcc.cc:205] Tried to move back new op lower bound from 7316528362963603456 to 7316528362776944640. Current Snapshot: MvccSnapshot[applied={T|T < 7316528362963603456}]
I20260809 07:50:07.393424 31419 mvcc.cc:205] Tried to move back new op lower bound from 7316528362963603456 to 7316528362776944640. Current Snapshot: MvccSnapshot[applied={T|T < 7316528362963603456}]
I20260809 07:50:07.394348 31420 mvcc.cc:205] Tried to move back new op lower bound from 7316528362963603456 to 7316528362776944640. Current Snapshot: MvccSnapshot[applied={T|T < 7316528362963603456}]
I20260809 07:50:07.404748 30846 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:50:07.404966 30846 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:50:07.407832 30846 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4d54448658d472c844e4006d2147d62: Sending DeleteTablet for 3 replicas of tablet cf28b2e69b9a4462b376529378e18c2c
I20260809 07:50:07.409417 31293 tablet_service.cc:1514] Processing DeleteTablet for tablet cf28b2e69b9a4462b376529378e18c2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:50:07 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:60080
I20260809 07:50:07.409420 31006 tablet_service.cc:1514] Processing DeleteTablet for tablet cf28b2e69b9a4462b376529378e18c2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:50:07 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:33036
I20260809 07:50:07.410017 31149 tablet_service.cc:1514] Processing DeleteTablet for tablet cf28b2e69b9a4462b376529378e18c2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:50:07 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:44146
I20260809 07:50:07.411547 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 30933
I20260809 07:50:07.418404 31428 tablet_replica.cc:330] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: stopping tablet replica
I20260809 07:50:07.418800 31428 raft_consensus.cc:2229] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:50:07.419186 31428 raft_consensus.cc:2258] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:50:07.430275 31428 ts_tablet_manager.cc:1905] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 07:50:07.432935 30832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:34247 (error 108)
I20260809 07:50:07.433992 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 31084
I20260809 07:50:07.435315 31428 ts_tablet_manager.cc:1918] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:50:07.435420 31428 log.cc:1196] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/cf28b2e69b9a4462b376529378e18c2c
I20260809 07:50:07.436185 31428 ts_tablet_manager.cc:1939] T cf28b2e69b9a4462b376529378e18c2c P d1310654c33b4835a3cb10ff9ec670e2: Deleting consensus metadata
I20260809 07:50:07.437721 30831 catalog_manager.cc:4841] TS d1310654c33b4835a3cb10ff9ec670e2 (127.18.71.67:35241): tablet cf28b2e69b9a4462b376529378e18c2c (table test-table [id=908292d81c614aa8a3c62d0ebe5251a1]) successfully deleted
I20260809 07:50:07.442392 31392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:07.442001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:34247 (local address 127.0.0.1:33054)
0809 07:50:07.442225 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.442299 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:34247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260809 07:50:07.442719 30832 catalog_manager.cc:4584] TS ca98381c68c3400f8b5013c47723b76a (127.18.71.65:34247): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf28b2e69b9a4462b376529378e18c2c: Network error: Client connection negotiation failed: client connection to 127.18.71.65:34247: connect: Connection refused (error 111)
W20260809 07:50:07.448755 31392 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:50:07.448112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35759 (local address 127.0.0.1:44154)
0809 07:50:07.448284 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:07.448299 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:07.448392 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:07.448642 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.66:35759: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 07:50:07.449535 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 31233
I20260809 07:50:07.462365 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 30813
I20260809 07:50:07.472280 18717 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 07:50:07.817207 18717 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 07:50:07.817386 18717 mini_ranger.cc:66] Stopping Ranger...
I20260809 07:50:08.162444 18717 mini_ranger.cc:68] Stopped Ranger
2026-08-09 07:50:08.162 UTC [30354] LOG: received smart shutdown request
2026-08-09 07:50:08.166 UTC [30354] LOG: background worker "logical replication launcher" (PID 30365) exited with exit code 1
2026-08-09 07:50:08.166 UTC [30358] LOG: shutting down
2026-08-09 07:50:08.340 UTC [30354] LOG: database system is shut down
2026-08-09T07:50:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47752 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:50:08 dist-test-slave-10pc krb5kdc[31440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:50:08 dist-test-slave-10pc krb5kdc[31440](info): set up 2 sockets
Aug 09 07:50:08 dist-test-slave-10pc krb5kdc[31440](info): commencing operation
krb5kdc: starting...
W20260809 07:50:09.548331 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:50:09 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:50:09Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/rangeradmin_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/rangeradmin_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/rangerlookup_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/rangerlookup_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
I20260809 07:50:09.732041 18717 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxTNb9d/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/postgres -l logfile start
2026-08-09 07:50:12.268 UTC [31471] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 07:50:12.268 UTC [31471] LOG: listening on IPv4 address "127.18.71.84", port 39453
2026-08-09 07:50:12.272 UTC [31471] LOG: listening on Unix socket "/tmp/.s.PGSQL.39453"
2026-08-09 07:50:12.294 UTC [31474] LOG: database system was shut down at 2026-08-09 07:50:10 UTC
2026-08-09 07:50:12.298 UTC [31471] LOG: database system is ready to accept connections
I20260809 07:50:13.285373 18717 mini_postgres.cc:96] Postgres bound to 39453
2026-08-09 07:50:13.293 UTC [31482] FATAL: database "slave" does not exist
127.18.71.84:39453 - accepting connections
I20260809 07:50:13.294184 18717 mini_ranger.cc:161] Starting Ranger...
I20260809 07:50:13.309734 18717 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 07:50:13.800933 18717 mini_ranger.cc:87] Created ranger Postgres database
I20260809 07:50:13.801033 18717 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-admin
2026-08-09 07:50:14,093 [I] DB FLAVOR :POSTGRES
2026-08-09 07:50:14,093 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 07:50:14,093 [I] Checking connection..
2026-08-09 07:50:14,094 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 07:50:14,358 [I] Checking connection passed.
2026-08-09 07:50:14,358 [I] --------- Verifying version history table ---------
2026-08-09 07:50:14,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 07:50:14,631 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 07:50:14,631 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 07:50:14,878 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 07:50:14,879 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 07:50:14,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 07:50:15.138 UTC [31552] WARNING: there is no transaction in progress
2026-08-09 07:50:15,145 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 07:50:15,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 07:50:15,396 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 07:50:15,396 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 07:50:15,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 07:50:15,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 07:50:15,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-10pc.c.gcp-upstream.internal','N') ;"
2026-08-09 07:50:16,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 07:50:16,412 [I] Table x_portal_user does not exist in database ranger
2026-08-09 07:50:16,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 07:50:16,675 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 07:50:16,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 07:50:16,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 07:50:17,169 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 07:50:17,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 07:50:18.122 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.138 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.154 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.169 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.185 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.204 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.259 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.265 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.278 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.287 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.297 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.311 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.320 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.326 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.333 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18.339 UTC [31666] WARNING: there is no transaction in progress
2026-08-09 07:50:18,942 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 07:50:18,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 07:50:19,184 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 07:50:19,184 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 07:50:19,441 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 07:50:19,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 07:50:19,705 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 07:50:19,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 07:50:19,964 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 07:50:20,223 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-10pc.c.gcp-upstream.internal';"
2026-08-09 07:50:20,465 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 07:50:20,466 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 07:50:20,729 [I] DB_PATCHES have already been applied
I20260809 07:50:20.735019 18717 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxTNb9d/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 07:50:20.735087 18717 mini_ranger.cc:193] Using host: 127.18.71.84
I20260809 07:50:20.739499 18717 mini_ranger.cc:227] Ranger admin URL: http://127.18.71.84:45501
Aug 09, 2026 7:50:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:50:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:50:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 7:50:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 7:50:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 7:50:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 7:50:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 07:50:21.916806 18717 mini_ranger.cc:160] Time spent starting Ranger: real 8.623s user 0.000s sys 0.011s
Aug 09, 2026 7:50:42 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 7:50:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 7:50:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 7:50:42 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 7:50:43 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 07:50:44.075039 18717 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.18.71.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.18.71.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/rangerkms_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.18.71.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/rangerkms_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.18.71.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.18.71.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.18.71.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/HTTP_127.18.71.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 07:50:44 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261844, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 07:50:44.184770 18717 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 07:50:44.194911 18717 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 07:50:45.004503 18717 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 07:50:45.004604 18717 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms
I20260809 07:50:45.004629 18717 mini_ranger_kms.cc:227] Using postgres at 127.18.71.84:39453
2026-08-09 07:50:45,179 [I] DB FLAVOR :POSTGRES
2026-08-09 07:50:45,180 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 07:50:45,180 [I] Checking connection
2026-08-09 07:50:45,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 07:50:45,417 [I] connection success
2026-08-09 07:50:45,418 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 07:50:45,418 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 07:50:45,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 07:50:45,656 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 07:50:45,656 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 07:50:45,656 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 07:50:45,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.18.71.84:39453/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 07:50:45,927 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 07:50:46.422159 18717 mini_ranger_kms.cc:310] Created kms service
I20260809 07:50:46.623871 18717 mini_ranger_kms.cc:326] Created kudu user
I20260809 07:50:46.667460 18717 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 07:50:46 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261844, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:47.060277 18717 mini_ranger_kms.cc:384] Added ranger policy
I20260809 07:50:47.060436 18717 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxTNb9d/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxTNb9d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 07:50:47.060501 18717 mini_ranger_kms.cc:240] Using host: 127.18.71.84
I20260809 07:50:47.065277 18717 mini_ranger_kms.cc:276] Ranger KMS PID: 31865
I20260809 07:50:47.065359 18717 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.18.71.84:46879
Aug 09, 2026 7:50:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 7:50:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 7:50:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 7:50:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 7:50:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 7:50:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 07:50:49.735543 18717 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.551s user 0.009s sys 0.151s
I20260809 07:50:49.735698 18717 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 07:50:49.735808 18717 mini_ranger_kms.cc:401] 127.18.71.84:46879/kms/v1/keys
Aug 09 07:50:50 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261850, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.71.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 07:50:50 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261850, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.71.84@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
Aug 09 07:50:51 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261850, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.18.71.84@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
Aug 09 07:50:53 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:50:53.423070 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:36215
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:36215
--ranger_config_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:50:53.549219 31925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:53.549510 31925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:53.549587 31925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:53.553622 31925 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:53.553681 31925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:53.553701 31925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:53.553718 31925 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:53.553733 31925 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:53.559286 31925 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
--ranger_kms_url=127.18.71.84:46879/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:36215
--ranger_config_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:36215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:53.560437 31925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:53.561612 31925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:53.570892 31930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:53.570953 31925 server_base.cc:1034] running on GCE node
W20260809 07:50:53.570894 31933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:53.571233 31931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:53.571805 31925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:53.573098 31925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:53.574299 31925 hybrid_clock.cc:648] HybridClock initialized: now 1786261853574263 us; error 54 us; skew 500 ppm
Aug 09 07:50:53 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261853, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:53.581133 31925 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:50:53.582132 31925 webserver.cc:460] Webserver started at http://127.18.71.126:32863/ using document root <none> and password file <none>
I20260809 07:50:53.582688 31925 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:53.582747 31925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:53.582974 31925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:53 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261853, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:53.632258 31925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "82c0a1288ea84b5e8208f404d2775849"
format_stamp: "Formatted at 2026-08-09 07:50:53 on dist-test-slave-10pc"
server_key: "826ee7cae88b129701e24659865bfa53"
server_key_iv: "f9eda812e89765fc10d38b3dc0a6de79"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:53.632974 31925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "82c0a1288ea84b5e8208f404d2775849"
format_stamp: "Formatted at 2026-08-09 07:50:53 on dist-test-slave-10pc"
server_key: "826ee7cae88b129701e24659865bfa53"
server_key_iv: "f9eda812e89765fc10d38b3dc0a6de79"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:53.636950 31925 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:50:53.652104 31940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:53.653563 31925 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:50:53.653721 31925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "82c0a1288ea84b5e8208f404d2775849"
format_stamp: "Formatted at 2026-08-09 07:50:53 on dist-test-slave-10pc"
server_key: "826ee7cae88b129701e24659865bfa53"
server_key_iv: "f9eda812e89765fc10d38b3dc0a6de79"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:53.653863 31925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:53.663868 31925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:53.666889 31925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:53.667096 31925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:53.675032 31925 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:36215
I20260809 07:50:53.675069 31996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:36215 every 8 connection(s)
I20260809 07:50:53.676055 31925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:53.679606 31998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:53.681432 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 31925
I20260809 07:50:53.681579 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:50:53.681892 18717 external_mini_cluster.cc:1428] Setting key a844cde0c2a138bd2bc86c73ac71d079
I20260809 07:50:53.687116 31998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Bootstrap starting.
I20260809 07:50:53.689874 31998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Neither blocks nor log segments found. Creating new log.
Aug 09 07:50:53 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:53.690774 31998 log.cc:824] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:53.692927 31998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: No bootstrap required, opened a new log
I20260809 07:50:53.696516 31998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:53.696797 31998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:53.696872 31998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c0a1288ea84b5e8208f404d2775849, State: Initialized, Role: FOLLOWER
I20260809 07:50:53.697388 31998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:53.697544 31998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:50:53.697573 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:53.683306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58080 (local address 127.18.71.126:36215)
0809 07:50:53.683707 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:53.683722 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:53.683758 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:50:53.684386 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:53.684403 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:53.684803 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:53.685156 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:53.685986 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:53.687008 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:53.687965 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:53.688421 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:53.688666 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:53.691261 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:53.691267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:53.691284 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:53.691313 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:53.694098 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:53.694636 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:53.694642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:53.694648 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:53.694793 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:53.695132 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:53.695135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:53.695137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:53.695665 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:53.695876 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:53.696200 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 07:50:53.696529 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":130,"threads_started":1}
I20260809 07:50:53.697638 31998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:50:53.697760 31998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:53.698653 31998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:53.699095 31998 leader_election.cc:302] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82c0a1288ea84b5e8208f404d2775849; no voters:
I20260809 07:50:53.699517 31998 leader_election.cc:288] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:50:53.699630 32004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:50:53.699887 32004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 LEADER]: Becoming Leader. State: Replica: 82c0a1288ea84b5e8208f404d2775849, State: Running, Role: LEADER
I20260809 07:50:53.700277 32004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:53.700871 31998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:53.701599 32004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82c0a1288ea84b5e8208f404d2775849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } } }
I20260809 07:50:53.701792 32004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:53.702059 32004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c0a1288ea84b5e8208f404d2775849. Latest consensus state: current_term: 1 leader_uuid: "82c0a1288ea84b5e8208f404d2775849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } } }
I20260809 07:50:53.702208 32004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:53.702519 32014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:50:53.706436 32014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:50:53.706861 31998 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 07:50:53.712350 32014 catalog_manager.cc:1347] Generated new cluster ID: 337c0b85cfb04b04bd9cbc9e2cab4033
I20260809 07:50:53.712414 32014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:50:53.726646 32014 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:50:53.728082 32014 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:50:53.736689 32014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Generated new TSK 0
I20260809 07:50:53.737696 32014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:50:54.500263 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36215
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:50:54.663762 32050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:54.664021 32050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:54.664063 32050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:54.668617 32050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:54.668766 32050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:54.668879 32050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:50:54.676784 32050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.18.71.84:46879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:54.678155 32050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:54.679399 32050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:54.692154 32058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:54.692487 32057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:54.692605 32060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:54.692796 32050 server_base.cc:1034] running on GCE node
I20260809 07:50:54.693202 32050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:54.693883 32050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:54.695056 32050 hybrid_clock.cc:648] HybridClock initialized: now 1786261854695025 us; error 56 us; skew 500 ppm
Aug 09 07:50:54 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:54.697961 32050 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:50:54.698959 32050 webserver.cc:460] Webserver started at http://127.18.71.65:35591/ using document root <none> and password file <none>
I20260809 07:50:54.699530 32050 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:54.699621 32050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:54.699829 32050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:54 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:54.714833 32050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "1ab944e718a24fffb95b180da95a84d1"
format_stamp: "Formatted at 2026-08-09 07:50:54 on dist-test-slave-10pc"
server_key: "a747a72e528cb13a71bcff16ad5dad14"
server_key_iv: "069c3e18097fed7ca5631db8fe7889ac"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:54.715620 32050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "1ab944e718a24fffb95b180da95a84d1"
format_stamp: "Formatted at 2026-08-09 07:50:54 on dist-test-slave-10pc"
server_key: "a747a72e528cb13a71bcff16ad5dad14"
server_key_iv: "069c3e18097fed7ca5631db8fe7889ac"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:54.724310 32050 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260809 07:50:54.741771 32067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:54.743536 32050 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260809 07:50:54.743685 32050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "1ab944e718a24fffb95b180da95a84d1"
format_stamp: "Formatted at 2026-08-09 07:50:54 on dist-test-slave-10pc"
server_key: "a747a72e528cb13a71bcff16ad5dad14"
server_key_iv: "069c3e18097fed7ca5631db8fe7889ac"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:54.743821 32050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:54.770035 32050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:54.780563 32050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:54.780804 32050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:54.781545 32050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:54.782625 32050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:54.782689 32050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:54.782743 32050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:54.782765 32050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:54.792721 32050 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:46373
I20260809 07:50:54.794097 32050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:54.794126 32194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:46373 every 8 connection(s)
I20260809 07:50:54.796737 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 32050
I20260809 07:50:54.796866 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:50:54.797217 18717 external_mini_cluster.cc:1428] Setting key 8d6d8d0478a69b105b96d53c8777873e
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Aug 09 07:50:54 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:54.837301 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:54.815492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:51755 (local address 127.18.71.126:36215)
0809 07:50:54.820634 (+ 5142us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:54.820639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:54.822330 (+ 1691us) server_negotiation.cc:407] Connection header received
0809 07:50:54.822396 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:54.822399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:54.822449 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:54.822569 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:54.823915 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:54.826731 (+ 2816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:54.828329 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:54.828779 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:54.828903 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:54.832532 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:54.832537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:54.832554 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:54.832593 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:54.834588 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:54.835298 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:54.835301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:54.835303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:54.835400 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:54.835875 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:54.835878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:54.835880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:54.836088 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:54.836195 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:54.836874 (+ 679us) server_negotiation.cc:299] Negotiation successful
0809 07:50:54.837007 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5057,"thread_start_us":79,"threads_started":1}
I20260809 07:50:54.838383 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:54.814282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36215 (local address 127.18.71.65:51755)
0809 07:50:54.820347 (+ 6065us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:54.820419 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:54.821836 (+ 1417us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:54.822787 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:54.822795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:54.823076 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:54.823699 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:54.823716 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:54.826911 (+ 3195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:54.826917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:54.828188 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:54.828199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:54.829113 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:54.829120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:54.829364 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:54.830139 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:54.830166 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:54.832247 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:54.834742 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:54.834750 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:54.834772 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:54.835164 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:54.835518 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:54.835524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:54.835528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:54.835755 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:54.836211 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:54.836219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:54.836584 (+ 365us) client_negotiation.cc:769] Sending connection context
0809 07:50:54.836819 (+ 235us) client_negotiation.cc:240] Negotiation successful
0809 07:50:54.837158 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5774,"thread_start_us":142,"threads_started":1}
I20260809 07:50:54.839851 32195 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36215
I20260809 07:50:54.840133 32195 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:54.840783 32195 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
I20260809 07:50:54.842525 31960 ts_manager.cc:194] Registered new tserver with Master: 1ab944e718a24fffb95b180da95a84d1 (127.18.71.65:46373)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:50:54.843743 31960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:51755
I20260809 07:50:54.845049 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36215
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:50:54 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 07:50:54.990149 32202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:54.990525 32202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:54.990597 32202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:54.997092 32202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:54.997176 32202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:54.997268 32202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:50:55.006083 32202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.18.71.84:46879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:55.007543 32202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:55.008971 32202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:55.021808 32210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:55.022150 32209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:55.022260 32213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:55.023209 32202 server_base.cc:1034] running on GCE node
I20260809 07:50:55.023712 32202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:55.024576 32202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:55.026834 32202 hybrid_clock.cc:648] HybridClock initialized: now 1786261855026754 us; error 95 us; skew 500 ppm
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:55.030596 32202 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:50:55.031781 32202 webserver.cc:460] Webserver started at http://127.18.71.66:34251/ using document root <none> and password file <none>
I20260809 07:50:55.032524 32202 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:55.032618 32202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:55.032853 32202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:55.051414 32202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "cece95fba9b04990a23d63dbfa0076ee"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5566df70f80a23d9288f86bf9d6e97be"
server_key_iv: "df37a78c800a8ccf1effb1f365d43bc3"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.054219 32202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "cece95fba9b04990a23d63dbfa0076ee"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5566df70f80a23d9288f86bf9d6e97be"
server_key_iv: "df37a78c800a8ccf1effb1f365d43bc3"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.059895 32202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:50:55.092912 32221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.094941 32202 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260809 07:50:55.095108 32202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "cece95fba9b04990a23d63dbfa0076ee"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5566df70f80a23d9288f86bf9d6e97be"
server_key_iv: "df37a78c800a8ccf1effb1f365d43bc3"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.095291 32202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:55.117730 32202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:55.121577 32202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:55.121794 32202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:55.122475 32202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:55.123349 32202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:55.123399 32202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.123462 32202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:55.123495 32202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.133631 32202 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:41033
I20260809 07:50:55.133715 32347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:41033 every 8 connection(s)
I20260809 07:50:55.134577 32202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:50:55.136482 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 32202
I20260809 07:50:55.136627 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:50:55.136994 18717 external_mini_cluster.cc:1428] Setting key 7f4cf55ad22009f302a5ac95b744bd94
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:55.160135 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:55.136679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:36011 (local address 127.18.71.126:36215)
0809 07:50:55.136842 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:55.136848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:55.143100 (+ 6252us) server_negotiation.cc:407] Connection header received
0809 07:50:55.144640 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:55.144647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:55.144714 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:55.144816 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:55.146329 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.147134 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.151965 (+ 4831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.152327 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.152405 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:55.155620 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:55.155625 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:55.155628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:55.155656 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:55.157471 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:55.158143 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:55.158146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:55.158148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:55.158242 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:55.158674 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:55.158679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:55.158684 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:55.158923 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:55.159011 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:55.159768 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 07:50:55.159922 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:50:55.162115 32351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:55.137189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36215 (local address 127.18.71.66:36011)
0809 07:50:55.142859 (+ 5670us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:55.142941 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:55.144330 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:55.145060 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:55.145071 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:55.145442 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:55.146168 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:55.146190 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.150388 (+ 4198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.150392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:55.151812 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:55.151824 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.152662 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.152666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:55.152889 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:55.153636 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:55.153660 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:55.155404 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:55.157622 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:55.157629 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:55.157650 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:55.158013 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:55.158352 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:55.158355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:55.158357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:55.158552 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:55.159098 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:55.159105 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:55.159463 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 07:50:55.160397 (+ 934us) client_negotiation.cc:240] Negotiation successful
0809 07:50:55.160825 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5368,"thread_start_us":149,"threads_started":1}
I20260809 07:50:55.163612 32349 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36215
I20260809 07:50:55.163898 32349 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:55.164602 32349 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:50:55.165885 31960 ts_manager.cc:194] Registered new tserver with Master: cece95fba9b04990a23d63dbfa0076ee (127.18.71.66:41033)
I20260809 07:50:55.166716 31960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:36011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:50:55.188171 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36215
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261854, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
W20260809 07:50:55.334676 32354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:55.335011 32354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:55.335078 32354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:55.341276 32354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:55.341367 32354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:55.341491 32354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:50:55.349093 32354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.18.71.84:46879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:36215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:55.350441 32354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:55.351886 32354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:55.364266 32362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:55.364266 32359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:55.364365 32360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:55.365420 32354 server_base.cc:1034] running on GCE node
I20260809 07:50:55.365850 32354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:55.366528 32354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:55.367713 32354 hybrid_clock.cc:648] HybridClock initialized: now 1786261855367682 us; error 42 us; skew 500 ppm
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:55.370687 32354 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:50:55.371635 32354 webserver.cc:460] Webserver started at http://127.18.71.67:40041/ using document root <none> and password file <none>
I20260809 07:50:55.372196 32354 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:55.372288 32354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:50:55.372566 32354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:55.384884 32354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "4e1b133c3a2f4e09a3c884cb59c78141"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5b88e3ba9d58c7ec5d67c5b97c01199b"
server_key_iv: "cc631f502beafeb6f7b49c79848856b2"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.385715 32354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "4e1b133c3a2f4e09a3c884cb59c78141"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5b88e3ba9d58c7ec5d67c5b97c01199b"
server_key_iv: "cc631f502beafeb6f7b49c79848856b2"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.395063 32354 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260809 07:50:55.413278 32370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.414954 32354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:55.415077 32354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "4e1b133c3a2f4e09a3c884cb59c78141"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5b88e3ba9d58c7ec5d67c5b97c01199b"
server_key_iv: "cc631f502beafeb6f7b49c79848856b2"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.415210 32354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:55.434696 32354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:55.439817 32354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:55.440096 32354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:55.440977 32354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:55.442293 32354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:55.442430 32354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.442564 32354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:55.442639 32354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.454461 32354 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:40819
I20260809 07:50:55.455825 32354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:50:55.457943 32488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:40819 every 8 connection(s)
I20260809 07:50:55.459029 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 32354
I20260809 07:50:55.459175 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:50:55.459494 18717 external_mini_cluster.cc:1428] Setting key 71a2c990b772edc6774def93562b33b1
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:55.477470 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:55.460666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:44557 (local address 127.18.71.126:36215)
0809 07:50:55.460834 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:55.460839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:55.463050 (+ 2211us) server_negotiation.cc:407] Connection header received
0809 07:50:55.464513 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:55.464517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:55.464584 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:55.464650 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:55.466664 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.467512 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.469437 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.469787 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.469890 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:55.473107 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:55.473111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:55.473114 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:55.473153 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:55.474944 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:55.475617 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:55.475620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:55.475622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:55.475709 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:55.476168 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:55.476171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:55.476173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:55.476380 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:55.476518 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:55.477193 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 07:50:55.477311 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 07:50:55.478727 32491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:55.461417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36215 (local address 127.18.71.67:44557)
0809 07:50:55.462828 (+ 1411us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:55.462910 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:55.464174 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:55.465181 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:55.465192 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:55.465610 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:55.466471 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:55.466495 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.467663 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.467668 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:55.469304 (+ 1636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:55.469317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:55.469978 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:55.469983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:55.470211 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:55.470964 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:55.470987 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:55.472920 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:55.475083 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:55.475090 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:55.475106 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:55.475502 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:55.475833 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:55.475836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:55.475839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:55.476055 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:55.476541 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:55.476551 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:55.476876 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 07:50:55.477137 (+ 261us) client_negotiation.cc:240] Negotiation successful
0809 07:50:55.477603 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"thread_start_us":184,"threads_started":1}
I20260809 07:50:55.480006 32489 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36215
I20260809 07:50:55.480268 32489 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:55.480902 32489 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
I20260809 07:50:55.482122 31960 ts_manager.cc:194] Registered new tserver with Master: 4e1b133c3a2f4e09a3c884cb59c78141 (127.18.71.67:40819)
I20260809 07:50:55.482841 31960 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:44557
I20260809 07:50:55.485524 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:50:55.486347 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 32050
I20260809 07:50:55.507596 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 32202
I20260809 07:50:55.519074 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 32354
I20260809 07:50:55.525633 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 31925
I20260809 07:50:55.534125 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:36215
--webserver_interface=127.18.71.126
--webserver_port=32863
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:36215
--ranger_config_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:50:55.649303 32493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:55.649559 32493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:55.649597 32493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:55.653729 32493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:50:55.653793 32493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:55.653813 32493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:55.653853 32493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:50:55.653885 32493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:50:55.659261 32493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:36215
--ranger_config_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:36215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=32863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:55.660382 32493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:55.661439 32493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:55.669948 32502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:55.669992 32500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:55.670135 32493 server_base.cc:1034] running on GCE node
W20260809 07:50:55.670300 32498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:55.670888 32493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:55.672001 32493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:55.673166 32493 hybrid_clock.cc:648] HybridClock initialized: now 1786261855673130 us; error 46 us; skew 500 ppm
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:55.676014 32493 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:50:55.676985 32493 webserver.cc:460] Webserver started at http://127.18.71.126:32863/ using document root <none> and password file <none>
I20260809 07:50:55.677536 32493 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:55.677632 32493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:55.722376 32493 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:50:55.724699 32510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.726052 32493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:50:55.726183 32493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "82c0a1288ea84b5e8208f404d2775849"
format_stamp: "Formatted at 2026-08-09 07:50:53 on dist-test-slave-10pc"
server_key: "826ee7cae88b129701e24659865bfa53"
server_key_iv: "f9eda812e89765fc10d38b3dc0a6de79"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.726744 32493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:55.740710 32493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:55.743634 32493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:55.743858 32493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:55.751566 32493 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:36215
I20260809 07:50:55.751596 32567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:36215 every 8 connection(s)
I20260809 07:50:55.752831 32493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:50:55.757166 32568 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:50:55.758793 32568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Bootstrap starting.
I20260809 07:50:55.761999 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 32493
I20260809 07:50:55.762494 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:46373
--local_ip_for_outbound_sockets=127.18.71.65
--tserver_master_addrs=127.18.71.126:36215
--webserver_port=35591
--webserver_interface=127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:50:55.776495 32568 log.cc:824] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:55.783205 32568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:50:55.783607 32568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Bootstrap complete.
I20260809 07:50:55.787904 32568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:55.788300 32568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c0a1288ea84b5e8208f404d2775849, State: Initialized, Role: FOLLOWER
I20260809 07:50:55.788812 32568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:55.788928 32568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:50:55.789009 32568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:50:55.789106 32568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:50:55.790944 32568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:55.791306 32568 leader_election.cc:302] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82c0a1288ea84b5e8208f404d2775849; no voters:
I20260809 07:50:55.791621 32568 leader_election.cc:288] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 07:50:55.791740 32572 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:50:55.791987 32572 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [term 2 LEADER]: Becoming Leader. State: Replica: 82c0a1288ea84b5e8208f404d2775849, State: Running, Role: LEADER
I20260809 07:50:55.792363 32572 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } }
I20260809 07:50:55.792871 32568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:50:55.793882 32574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82c0a1288ea84b5e8208f404d2775849. Latest consensus state: current_term: 2 leader_uuid: "82c0a1288ea84b5e8208f404d2775849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } } }
I20260809 07:50:55.794018 32574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:55.794346 32581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:50:55.794346 32573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "82c0a1288ea84b5e8208f404d2775849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82c0a1288ea84b5e8208f404d2775849" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 36215 } } }
I20260809 07:50:55.794523 32573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849 [sys.catalog]: This master's current role is: LEADER
I20260809 07:50:55.797828 32581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:50:55.798336 32568 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 07:50:55.798514 32581 catalog_manager.cc:1259] Loaded cluster ID: 337c0b85cfb04b04bd9cbc9e2cab4033
I20260809 07:50:55.798564 32581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:50:55.800909 32581 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:50:55.801563 32581 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Loaded TSK: 0
I20260809 07:50:55.802734 32581 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 07:50:55.904084 32570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:55.904356 32570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:55.904452 32570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:55.908504 32570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:55.908591 32570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:55.908697 32570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:50:55.915499 32570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:46373
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=35591
--tserver_master_addrs=127.18.71.126:36215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:55.916991 32570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:55.918377 32570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:55.928383 32605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:55.928489 32608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:55.928656 32606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:55.928925 32570 server_base.cc:1034] running on GCE node
I20260809 07:50:55.929370 32570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:55.930137 32570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:55.931331 32570 hybrid_clock.cc:648] HybridClock initialized: now 1786261855931302 us; error 38 us; skew 500 ppm
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:55.934607 32570 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:50:55.935585 32570 webserver.cc:460] Webserver started at http://127.18.71.65:35591/ using document root <none> and password file <none>
I20260809 07:50:55.936115 32570 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:55.936182 32570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 07:50:55 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:55.978355 32570 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.003s sys 0.000s
I20260809 07:50:55.983664 32616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:55.985495 32570 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260809 07:50:55.985661 32570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "1ab944e718a24fffb95b180da95a84d1"
format_stamp: "Formatted at 2026-08-09 07:50:54 on dist-test-slave-10pc"
server_key: "a747a72e528cb13a71bcff16ad5dad14"
server_key_iv: "069c3e18097fed7ca5631db8fe7889ac"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:55.986481 32570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:56.020133 32570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:56.048691 32570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:56.049045 32570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:56.049995 32570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:56.051240 32570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:56.051316 32570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.051380 32570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:56.051409 32570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.074976 32570 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:46373
I20260809 07:50:56.075035 32737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:46373 every 8 connection(s)
I20260809 07:50:56.076364 32570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:50:56.086629 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 32570
I20260809 07:50:56.087235 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:41033
--local_ip_for_outbound_sockets=127.18.71.66
--tserver_master_addrs=127.18.71.126:36215
--webserver_port=34251
--webserver_interface=127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:50:56 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261855, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:56.113415 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:56.084110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36215 (local address 127.18.71.65:36325)
0809 07:50:56.094139 (+ 10029us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:56.094225 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:56.095562 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:56.097696 (+ 2134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:56.097707 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:56.098096 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:56.098814 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:56.098837 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.100977 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.100984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:56.102592 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:56.102606 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.104095 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.104101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:56.104339 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:56.105140 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:56.105167 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:56.107154 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:56.109866 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:56.109872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:56.109887 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:56.110226 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:56.110640 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:56.110643 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:56.110646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:56.110804 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:56.111545 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:56.111554 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:56.111846 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 07:50:56.112051 (+ 205us) client_negotiation.cc:240] Negotiation successful
0809 07:50:56.112365 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9726,"thread_start_us":133,"threads_started":1}
I20260809 07:50:56.115170 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:56.084014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:36325 (local address 127.18.71.126:36215)
0809 07:50:56.092786 (+ 8772us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:56.092812 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:56.094393 (+ 1581us) server_negotiation.cc:407] Connection header received
0809 07:50:56.096146 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:56.096164 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:56.096684 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:56.097066 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:56.099030 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.100788 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.102769 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.103611 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.103823 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:56.107415 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:56.107426 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:56.107447 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:56.107486 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:56.109731 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.110374 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.110380 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.110385 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.110525 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.110908 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.110911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.110913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.111369 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:56.111482 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:56.113951 (+ 2469us) server_negotiation.cc:299] Negotiation successful
0809 07:50:56.114235 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8527,"thread_start_us":104,"threads_started":1}
I20260809 07:50:56.118564 32738 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36215
I20260809 07:50:56.118814 32738 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:56.119395 32738 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
W20260809 07:50:56.121713 32738 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:36215 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:50:56.121829 32738 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:56.121999 32738 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
I20260809 07:50:56.122931 32738 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:56.123092 32738 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
W20260809 07:50:56.123920 32738 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 07:50:56.255951 32742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:56.256264 32742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:56.256359 32742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:56.261340 32742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:56.261438 32742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:56.261557 32742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:50:56.269114 32742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:41033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=34251
--tserver_master_addrs=127.18.71.126:36215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:56.270530 32742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:56.271931 32742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:56.283993 32748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:56.284021 32747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:56.284005 32750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:56.285271 32742 server_base.cc:1034] running on GCE node
I20260809 07:50:56.285740 32742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:56.286506 32742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:56.287685 32742 hybrid_clock.cc:648] HybridClock initialized: now 1786261856287662 us; error 31 us; skew 500 ppm
Aug 09 07:50:56 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:56.291572 32742 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:50:56.292819 32742 webserver.cc:460] Webserver started at http://127.18.71.66:34251/ using document root <none> and password file <none>
I20260809 07:50:56.293653 32742 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:56.293943 32742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 07:50:56 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:56.337643 32742 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.004s sys 0.000s
I20260809 07:50:56.345115 32758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.347985 32742 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260809 07:50:56.348141 32742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "cece95fba9b04990a23d63dbfa0076ee"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5566df70f80a23d9288f86bf9d6e97be"
server_key_iv: "df37a78c800a8ccf1effb1f365d43bc3"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:56.348994 32742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:56.370852 32742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:56.375228 32742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:56.375543 32742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:56.376333 32742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:56.377583 32742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:56.377698 32742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.377807 32742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:56.377882 32742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.390453 32742 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:41033
I20260809 07:50:56.391994 32742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:50:56.392442 411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:41033 every 8 connection(s)
I20260809 07:50:56.398182 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 32742
I20260809 07:50:56.398680 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.18.71.84:46879/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:40819
--local_ip_for_outbound_sockets=127.18.71.67
--tserver_master_addrs=127.18.71.126:36215
--webserver_port=40041
--webserver_interface=127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:50:56 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:56.419974 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:56.397210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:36337 (local address 127.18.71.126:36215)
0809 07:50:56.397343 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:56.397349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:56.401260 (+ 3911us) server_negotiation.cc:407] Connection header received
0809 07:50:56.406550 (+ 5290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:56.406555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:56.406638 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:56.406741 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:56.408537 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.409432 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.411338 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.411746 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.411836 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:56.415285 (+ 3449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:56.415289 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:56.415292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:56.415321 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:56.417270 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.417977 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.417996 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.418001 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.418100 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.418570 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.418574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.418576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.418807 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:56.418892 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:56.419640 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 07:50:56.419789 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 07:50:56.422166 414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:56.396042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36215 (local address 127.18.71.66:36337)
0809 07:50:56.401069 (+ 5027us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:56.401131 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:56.406247 (+ 5116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:56.407111 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:56.407121 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:56.407507 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:56.408288 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:56.408312 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.409614 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.409618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:56.411176 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:56.411188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.412089 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.412094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:56.412309 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:56.413127 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:56.413150 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:56.415053 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:56.417426 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:56.417432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:56.417449 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:56.417822 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:56.418233 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:56.418237 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:56.418239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:56.418445 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:56.418962 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:56.418969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:56.419330 (+ 361us) client_negotiation.cc:769] Sending connection context
0809 07:50:56.420505 (+ 1175us) client_negotiation.cc:240] Negotiation successful
0809 07:50:56.420874 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4761,"thread_start_us":154,"threads_started":1}
I20260809 07:50:56.423967 412 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36215
I20260809 07:50:56.424240 412 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:56.424829 412 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
W20260809 07:50:56.426560 412 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:36215 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 07:50:56.426724 412 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:56.426927 412 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
I20260809 07:50:56.427837 412 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:56.428049 412 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
W20260809 07:50:56.428977 412 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 07:50:56.587731 415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:50:56.588117 415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:50:56.588201 415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:50:56.594574 415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:50:56.594655 415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:50:56.594758 415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:50:56.602895 415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.18.71.84:46879/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:40819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=40041
--tserver_master_addrs=127.18.71.126:36215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:50:56.604334 415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:50:56.605831 415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:50:56.618383 422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:50:56.619053 424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:56.619293 415 server_base.cc:1034] running on GCE node
W20260809 07:50:56.620594 420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:50:56.621079 415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:50:56.621769 415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:50:56.625049 415 hybrid_clock.cc:648] HybridClock initialized: now 1786261856625016 us; error 49 us; skew 500 ppm
Aug 09 07:50:56 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:56.628608 415 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:50:56.629814 415 webserver.cc:460] Webserver started at http://127.18.71.67:40041/ using document root <none> and password file <none>
I20260809 07:50:56.630640 415 fs_manager.cc:359] Metadata directory not provided
I20260809 07:50:56.630813 415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 07:50:56 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:56.662233 415 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.003s sys 0.000s
I20260809 07:50:56.666340 434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.668239 415 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:50:56.668437 415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "4e1b133c3a2f4e09a3c884cb59c78141"
format_stamp: "Formatted at 2026-08-09 07:50:55 on dist-test-slave-10pc"
server_key: "5b88e3ba9d58c7ec5d67c5b97c01199b"
server_key_iv: "cc631f502beafeb6f7b49c79848856b2"
server_key_version: "kuduclusterkey@0"
I20260809 07:50:56.669330 415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:50:56.687455 415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:50:56.691231 415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:50:56.691480 415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:50:56.692322 415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:50:56.693663 415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:50:56.693740 415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.693794 415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:50:56.693825 415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:50:56.705694 415 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:40819
I20260809 07:50:56.705780 566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:40819 every 8 connection(s)
I20260809 07:50:56.706789 415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:50:56.709815 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 415
Aug 09 07:50:56 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261856, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:50:56.721329 571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:56.710914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.18.71.126:36215)
0809 07:50:56.711180 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:56.711188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:56.711201 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:50:56.711250 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:56.711253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:56.711302 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:56.712370 (+ 1068us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:56.712509 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.713345 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.714552 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.714907 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.715007 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:56.716881 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:56.716885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:56.716888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:56.716908 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:56.718757 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.719259 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.719262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.719264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.719359 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.719665 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.719668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.719670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.719944 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:56.720185 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:56.720934 (+ 749us) server_negotiation.cc:299] Negotiation successful
0809 07:50:56.721079 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":195,"thread_start_us":118,"threads_started":1}
I20260809 07:50:56.723768 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:56.708963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:55021 (local address 127.18.71.126:36215)
0809 07:50:56.709094 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:56.709099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:56.710210 (+ 1111us) server_negotiation.cc:407] Connection header received
0809 07:50:56.712423 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:56.712426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:56.712499 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:56.712548 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:56.714134 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.714843 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.716267 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.716581 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.716699 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:56.719637 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:56.719643 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:56.719649 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:56.719676 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:56.721375 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.722033 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.722037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.722038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.722132 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:56.722486 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:56.722489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:56.722491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:56.722661 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:56.722760 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:56.723519 (+ 759us) server_negotiation.cc:299] Negotiation successful
0809 07:50:56.723629 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51,"spinlock_wait_cycles":768}
I20260809 07:50:56.724622 569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:56.709507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:36215 (local address 127.18.71.67:55021)
0809 07:50:56.710035 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:56.710090 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:56.712186 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:56.712937 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:56.712945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:56.713287 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:50:56.713956 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:56.713976 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.714994 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.714997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:56.716135 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:56.716145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:56.716746 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:56.716751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:56.716962 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:56.717751 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:50:56.717778 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:50:56.719466 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:50:56.721513 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:56.721519 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:56.721534 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:56.721880 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:56.722238 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:50:56.722240 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:50:56.722241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:50:56.722373 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:50:56.722780 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:50:56.722787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:50:56.723102 (+ 315us) client_negotiation.cc:769] Sending connection context
0809 07:50:56.723314 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 07:50:56.723656 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":151,"threads_started":1}
I20260809 07:50:56.725886 567 heartbeater.cc:347] Connected to a master server at 127.18.71.126:36215
I20260809 07:50:56.726150 567 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:56.726649 567 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
I20260809 07:50:56.727983 32526 ts_manager.cc:194] Registered new tserver with Master: 4e1b133c3a2f4e09a3c884cb59c78141 (127.18.71.67:40819)
I20260809 07:50:56.729425 32526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:55021
Aug 09 07:50:57 dist-test-slave-10pc krb5kdc[31440](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786261857, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:50:57.124218 32738 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:57.124588 32738 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
I20260809 07:50:57.125380 32526 ts_manager.cc:194] Registered new tserver with Master: 1ab944e718a24fffb95b180da95a84d1 (127.18.71.65:46373)
I20260809 07:50:57.126279 32526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:36325
Aug 09 07:50:57 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261857, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
I20260809 07:50:57.429234 412 heartbeater.cc:464] Registering TS with master...
I20260809 07:50:57.429539 412 heartbeater.cc:511] Master 127.18.71.126:36215 requested a full tablet report, sending...
I20260809 07:50:57.430384 32526 ts_manager.cc:194] Registered new tserver with Master: cece95fba9b04990a23d63dbfa0076ee (127.18.71.66:41033)
I20260809 07:50:57.431181 32526 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:36337
I20260809 07:50:57.439425 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:50:57.463080 581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.441748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33218 (local address 127.18.71.126:36215)
0809 07:50:57.452691 (+ 10943us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:57.452695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:57.452709 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:50:57.452755 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:57.452757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:57.452800 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:57.452909 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:50:57.453845 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.454786 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.456104 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.456429 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.456537 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.458509 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:50:57.458513 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:50:57.458516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:50:57.458544 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:50:57.460799 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:57.461300 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:57.461303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:57.461305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:57.461395 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:50:57.461707 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:50:57.461710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:50:57.461712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:50:57.461925 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:50:57.462028 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:57.462767 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 07:50:57.462897 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10822,"thread_start_us":107,"threads_started":1}
I20260809 07:50:57.467674 32526 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33218:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:50:57.472725 32526 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:50:57.510504 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.487235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:41033 (local address 127.0.0.1:57628)
0809 07:50:57.492063 (+ 4828us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:57.492116 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:57.492248 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:57.500519 (+ 8271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:57.500522 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:57.500539 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:57.500622 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.500627 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.507259 (+ 6632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.507263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.508623 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.508634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.509924 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.509928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.510061 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.510118 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 07:50:57.510222 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 07:50:57.510345 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4569,"thread_start_us":90,"threads_started":1}
I20260809 07:50:57.510988 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.488259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.18.71.66:41033)
0809 07:50:57.500057 (+ 11798us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:57.500063 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:57.500086 (+ 23us) server_negotiation.cc:407] Connection header received
0809 07:50:57.500157 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:57.500161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:57.500332 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:57.503881 (+ 3549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:57.504037 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.505166 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.508787 (+ 3621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.509597 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.509718 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.509857 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:57.510745 (+ 888us) server_negotiation.cc:299] Negotiation successful
0809 07:50:57.510854 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11686,"thread_start_us":77,"threads_started":1}
I20260809 07:50:57.511673 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.490294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:40819 (local address 127.0.0.1:46308)
0809 07:50:57.493208 (+ 2914us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:57.493220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:57.493298 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:57.495937 (+ 2639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:57.495941 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:57.495957 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:57.496050 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.496056 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.501949 (+ 5893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.501952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.503744 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.503753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.511201 (+ 7448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.511205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.511333 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.511370 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 07:50:57.511451 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 07:50:57.511536 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2857,"thread_start_us":101,"threads_started":1}
I20260809 07:50:57.512866 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.489198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:46373 (local address 127.0.0.1:60718)
0809 07:50:57.492640 (+ 3442us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:57.492652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:57.492766 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:57.495197 (+ 2431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:57.495204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:57.495226 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:57.495355 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.495361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.497488 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.497492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.499773 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.499785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.512442 (+ 12657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.512445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.512566 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.512588 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 07:50:57.512674 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 07:50:57.512745 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3379,"thread_start_us":85,"threads_started":1}
I20260809 07:50:57.513305 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.489606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.18.71.65:46373)
0809 07:50:57.493418 (+ 3812us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:57.493426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:57.493461 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:50:57.493535 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:57.493540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:57.493745 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:57.495021 (+ 1276us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:57.496172 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.497350 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.505300 (+ 7950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.512126 (+ 6826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.512236 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.512372 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:57.513052 (+ 680us) server_negotiation.cc:299] Negotiation successful
0809 07:50:57.513162 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3298,"thread_start_us":94,"threads_started":1}
I20260809 07:50:57.514508 325 tablet_service.cc:1467] Processing CreateTablet for tablet f5120184f3c34ed881abd4385bf95725 (DEFAULT_TABLE table=test-table [id=8dc77c6a1bea4ab48b907df749965742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:50:57.515726 325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5120184f3c34ed881abd4385bf95725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:57.516162 32667 tablet_service.cc:1467] Processing CreateTablet for tablet f5120184f3c34ed881abd4385bf95725 (DEFAULT_TABLE table=test-table [id=8dc77c6a1bea4ab48b907df749965742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:50:57.517225 32667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5120184f3c34ed881abd4385bf95725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:57.520931 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.490726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46308 (local address 127.18.71.67:40819)
0809 07:50:57.493975 (+ 3249us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:57.493982 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:57.494002 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:50:57.494074 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:57.494079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:57.495668 (+ 1589us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:57.495811 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:57.500742 (+ 4931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.501817 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.505939 (+ 4122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.506847 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.506971 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.507169 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:57.511531 (+ 4362us) server_negotiation.cc:299] Negotiation successful
0809 07:50:57.511643 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3151,"thread_start_us":86,"threads_started":1}
I20260809 07:50:57.524044 495 tablet_service.cc:1467] Processing CreateTablet for tablet f5120184f3c34ed881abd4385bf95725 (DEFAULT_TABLE table=test-table [id=8dc77c6a1bea4ab48b907df749965742]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:50:57.525202 495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5120184f3c34ed881abd4385bf95725. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:50:57.525702 594 tablet_bootstrap.cc:492] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1: Bootstrap starting.
I20260809 07:50:57.528128 594 tablet_bootstrap.cc:654] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:57.529083 594 log.cc:824] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:57.531836 594 tablet_bootstrap.cc:492] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1: No bootstrap required, opened a new log
I20260809 07:50:57.532032 594 ts_tablet_manager.cc:1397] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 07:50:57.533838 597 tablet_bootstrap.cc:492] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Bootstrap starting.
I20260809 07:50:57.536175 597 tablet_bootstrap.cc:654] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:57.536056 594 raft_consensus.cc:348] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.536384 594 raft_consensus.cc:374] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:57.536482 594 raft_consensus.cc:729] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab944e718a24fffb95b180da95a84d1, State: Initialized, Role: FOLLOWER
I20260809 07:50:57.536957 594 consensus_queue.cc:260] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.537060 597 log.cc:824] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:57.537642 32738 heartbeater.cc:503] Master 127.18.71.126:36215 was elected leader, sending a full tablet report...
I20260809 07:50:57.537765 596 tablet_bootstrap.cc:492] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Bootstrap starting.
I20260809 07:50:57.538053 594 ts_tablet_manager.cc:1428] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:50:57.539996 596 tablet_bootstrap.cc:654] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Neither blocks nor log segments found. Creating new log.
I20260809 07:50:57.540267 597 tablet_bootstrap.cc:492] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: No bootstrap required, opened a new log
I20260809 07:50:57.540526 597 ts_tablet_manager.cc:1397] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 07:50:57.540988 596 log.cc:824] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Log is configured to *not* fsync() on all Append() calls
I20260809 07:50:57.543660 596 tablet_bootstrap.cc:492] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: No bootstrap required, opened a new log
I20260809 07:50:57.543859 596 ts_tablet_manager.cc:1397] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:50:57.545625 597 raft_consensus.cc:348] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.545918 597 raft_consensus.cc:374] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:57.545979 597 raft_consensus.cc:729] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e1b133c3a2f4e09a3c884cb59c78141, State: Initialized, Role: FOLLOWER
I20260809 07:50:57.546451 597 consensus_queue.cc:260] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.547665 597 ts_tablet_manager.cc:1428] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 07:50:57.547821 596 raft_consensus.cc:348] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.548066 567 heartbeater.cc:503] Master 127.18.71.126:36215 was elected leader, sending a full tablet report...
I20260809 07:50:57.548081 596 raft_consensus.cc:374] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:50:57.548147 596 raft_consensus.cc:729] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cece95fba9b04990a23d63dbfa0076ee, State: Initialized, Role: FOLLOWER
I20260809 07:50:57.548588 596 consensus_queue.cc:260] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.549291 412 heartbeater.cc:503] Master 127.18.71.126:36215 was elected leader, sending a full tablet report...
I20260809 07:50:57.549584 596 ts_tablet_manager.cc:1428] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260809 07:50:57.584858 32739 tablet.cc:2369] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:50:57.644915 413 tablet.cc:2369] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 07:50:57 dist-test-slave-10pc krb5kdc[31440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261857, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for HTTP/127.18.71.84@KRBTEST.COM
W20260809 07:50:57.709488 568 tablet.cc:2369] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:50:57.718137 602 raft_consensus.cc:482] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:50:57.718331 602 raft_consensus.cc:504] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.719671 602 leader_election.cc:288] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cece95fba9b04990a23d63dbfa0076ee (127.18.71.66:41033), 1ab944e718a24fffb95b180da95a84d1 (127.18.71.65:46373)
I20260809 07:50:57.727080 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.720615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:53947 (local address 127.18.71.66:41033)
0809 07:50:57.720760 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:57.720764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:57.722491 (+ 1727us) server_negotiation.cc:407] Connection header received
0809 07:50:57.722719 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:57.722723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:57.722771 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:57.722820 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:57.723235 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.724009 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.725761 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.726378 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.726473 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.726526 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:57.726858 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 07:50:57.726954 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 07:50:57.727625 591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.720864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:38159 (local address 127.18.71.65:46373)
0809 07:50:57.721083 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:57.721087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:57.723278 (+ 2191us) server_negotiation.cc:407] Connection header received
0809 07:50:57.723474 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:57.723478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:57.723527 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:57.723575 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:50:57.723954 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.724736 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.726355 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.726933 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.727047 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.727097 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:57.727431 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 07:50:57.727509 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 07:50:57.728003 606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.720716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:46373 (local address 127.18.71.67:38159)
0809 07:50:57.723188 (+ 2472us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:57.723203 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:57.723348 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:57.723710 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:57.723713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:57.723729 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:57.723823 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.723829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.724888 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.724891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.726202 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.726213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.727196 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.727199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.727288 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.727309 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:50:57.727766 (+ 457us) client_negotiation.cc:240] Negotiation successful
0809 07:50:57.727839 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2406,"mutex_wait_us":12,"thread_start_us":116,"threads_started":1}
I20260809 07:50:57.728546 607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.720073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:41033 (local address 127.18.71.67:53947)
0809 07:50:57.722395 (+ 2322us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:57.722411 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:50:57.722585 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:50:57.722970 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:50:57.722973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:50:57.722991 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:50:57.723098 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.723104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.724196 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.724199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.725612 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:50:57.725624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.726631 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.726635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:50:57.726724 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.726745 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:50:57.727986 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0809 07:50:57.728064 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2249,"mutex_wait_us":30,"thread_start_us":121,"threads_started":1}
I20260809 07:50:57.728787 32689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5120184f3c34ed881abd4385bf95725" candidate_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ab944e718a24fffb95b180da95a84d1" is_pre_election: true
I20260809 07:50:57.729138 32689 raft_consensus.cc:2454] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e1b133c3a2f4e09a3c884cb59c78141 in term 0.
I20260809 07:50:57.729259 359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5120184f3c34ed881abd4385bf95725" candidate_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cece95fba9b04990a23d63dbfa0076ee" is_pre_election: true
I20260809 07:50:57.729532 359 raft_consensus.cc:2454] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e1b133c3a2f4e09a3c884cb59c78141 in term 0.
I20260809 07:50:57.729674 438 leader_election.cc:302] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ab944e718a24fffb95b180da95a84d1, 4e1b133c3a2f4e09a3c884cb59c78141; no voters:
I20260809 07:50:57.729970 602 raft_consensus.cc:2781] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:50:57.730048 602 raft_consensus.cc:482] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:50:57.730090 602 raft_consensus.cc:3037] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:57.731417 602 raft_consensus.cc:504] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.731791 602 leader_election.cc:288] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [CANDIDATE]: Term 1 election: Requested vote from peers cece95fba9b04990a23d63dbfa0076ee (127.18.71.66:41033), 1ab944e718a24fffb95b180da95a84d1 (127.18.71.65:46373)
I20260809 07:50:57.732146 359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5120184f3c34ed881abd4385bf95725" candidate_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cece95fba9b04990a23d63dbfa0076ee"
I20260809 07:50:57.732265 359 raft_consensus.cc:3037] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:57.732287 32689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5120184f3c34ed881abd4385bf95725" candidate_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ab944e718a24fffb95b180da95a84d1"
I20260809 07:50:57.732403 32689 raft_consensus.cc:3037] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:50:57.733618 359 raft_consensus.cc:2454] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e1b133c3a2f4e09a3c884cb59c78141 in term 1.
I20260809 07:50:57.733678 32689 raft_consensus.cc:2454] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e1b133c3a2f4e09a3c884cb59c78141 in term 1.
I20260809 07:50:57.733995 437 leader_election.cc:302] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e1b133c3a2f4e09a3c884cb59c78141, cece95fba9b04990a23d63dbfa0076ee; no voters:
I20260809 07:50:57.734160 602 raft_consensus.cc:2781] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:50:57.734397 602 raft_consensus.cc:686] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 1 LEADER]: Becoming Leader. State: Replica: 4e1b133c3a2f4e09a3c884cb59c78141, State: Running, Role: LEADER
I20260809 07:50:57.734756 602 consensus_queue.cc:237] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } }
I20260809 07:50:57.737962 32526 catalog_manager.cc:5495] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e1b133c3a2f4e09a3c884cb59c78141 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e1b133c3a2f4e09a3c884cb59c78141" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40819 } health_report { overall_health: HEALTHY } } }
I20260809 07:50:57.800793 593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.795630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46312 (local address 127.18.71.67:40819)
0809 07:50:57.795766 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:50:57.795771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:50:57.795785 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:50:57.795885 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:50:57.795888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:50:57.795981 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:50:57.796117 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:50:57.796546 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.797339 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.798656 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:50:57.798996 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:50:57.799085 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:50:57.799509 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:50:57.800062 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:50:57.800173 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 07:50:57.800573 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 07:50:57.800635 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 07:50:57.808506 32689 raft_consensus.cc:1261] T f5120184f3c34ed881abd4385bf95725 P 1ab944e718a24fffb95b180da95a84d1 [term 1 FOLLOWER]: Refusing update from remote peer 4e1b133c3a2f4e09a3c884cb59c78141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:57.809355 602 consensus_queue.cc:1035] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ab944e718a24fffb95b180da95a84d1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 46373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:57.816650 359 raft_consensus.cc:1261] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 1 FOLLOWER]: Refusing update from remote peer 4e1b133c3a2f4e09a3c884cb59c78141: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:50:57.818187 608 consensus_queue.cc:1035] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cece95fba9b04990a23d63dbfa0076ee" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:50:57.819612 612 mvcc.cc:205] Tried to move back new op lower bound from 7316528569576083456 to 7316528569284358144. Current Snapshot: MvccSnapshot[applied={T|T < 7316528569576083456}]
I20260809 07:50:57.827826 613 mvcc.cc:205] Tried to move back new op lower bound from 7316528569576083456 to 7316528569284358144. Current Snapshot: MvccSnapshot[applied={T|T < 7316528569576083456}]
I20260809 07:50:57.833580 616 mvcc.cc:205] Tried to move back new op lower bound from 7316528569576083456 to 7316528569284358144. Current Snapshot: MvccSnapshot[applied={T|T < 7316528569576083456}]
I20260809 07:50:57.839897 32527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:50:57.840070 32527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33218:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:50:57.842690 32527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82c0a1288ea84b5e8208f404d2775849: Sending DeleteTablet for 3 replicas of tablet f5120184f3c34ed881abd4385bf95725
I20260809 07:50:57.844347 325 tablet_service.cc:1514] Processing DeleteTablet for tablet f5120184f3c34ed881abd4385bf95725 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:50:57 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:57628
I20260809 07:50:57.845113 32667 tablet_service.cc:1514] Processing DeleteTablet for tablet f5120184f3c34ed881abd4385bf95725 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:50:57 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:60718
I20260809 07:50:57.845607 495 tablet_service.cc:1514] Processing DeleteTablet for tablet f5120184f3c34ed881abd4385bf95725 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:50:57 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:46308
I20260809 07:50:57.846669 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 32570
I20260809 07:50:57.854655 621 tablet_replica.cc:330] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: stopping tablet replica
I20260809 07:50:57.855041 621 raft_consensus.cc:2229] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:50:57.855991 621 raft_consensus.cc:2258] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:50:57.866945 623 tablet_replica.cc:330] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: stopping tablet replica
I20260809 07:50:57.867411 623 raft_consensus.cc:2229] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:50:57.868023 623 raft_consensus.cc:2258] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:50:57.869894 621 ts_tablet_manager.cc:1905] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:50:57.872819 621 ts_tablet_manager.cc:1918] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:50:57.872927 621 log.cc:1196] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/f5120184f3c34ed881abd4385bf95725
I20260809 07:50:57.872915 623 ts_tablet_manager.cc:1905] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:50:57.875409 623 ts_tablet_manager.cc:1918] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:50:57.875530 623 log.cc:1196] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/f5120184f3c34ed881abd4385bf95725
I20260809 07:50:57.876832 621 ts_tablet_manager.cc:1939] T f5120184f3c34ed881abd4385bf95725 P cece95fba9b04990a23d63dbfa0076ee: Deleting consensus metadata
I20260809 07:50:57.878240 32513 catalog_manager.cc:4841] TS cece95fba9b04990a23d63dbfa0076ee (127.18.71.66:41033): tablet f5120184f3c34ed881abd4385bf95725 (table test-table [id=8dc77c6a1bea4ab48b907df749965742]) successfully deleted
I20260809 07:50:57.878538 623 ts_tablet_manager.cc:1939] T f5120184f3c34ed881abd4385bf95725 P 4e1b133c3a2f4e09a3c884cb59c78141: Deleting consensus metadata
I20260809 07:50:57.879803 32511 catalog_manager.cc:4841] TS 4e1b133c3a2f4e09a3c884cb59c78141 (127.18.71.67:40819): tablet f5120184f3c34ed881abd4385bf95725 (table test-table [id=8dc77c6a1bea4ab48b907df749965742]) successfully deleted
W20260809 07:50:57.882869 32514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:46373 (error 108)
I20260809 07:50:57.883930 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 32742
I20260809 07:50:57.893635 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 415
I20260809 07:50:57.905105 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 32493
I20260809 07:50:57.905388 590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:50:57.905057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:46373 (local address 127.0.0.1:60732)
0809 07:50:57.905212 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 07:50:57.905300 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:46373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 07:50:57.914844 18717 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 07:50:58.260195 18717 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 07:50:58.260330 18717 mini_ranger.cc:66] Stopping Ranger...
I20260809 07:50:58.605898 18717 mini_ranger.cc:68] Stopped Ranger
2026-08-09 07:50:58.606 UTC [31471] LOG: received smart shutdown request
2026-08-09 07:50:58.610 UTC [31471] LOG: background worker "logical replication launcher" (PID 31480) exited with exit code 1
2026-08-09 07:50:58.610 UTC [31475] LOG: shutting down
2026-08-09 07:50:58.770 UTC [31471] LOG: database system is shut down
2026-08-09T07:50:58Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50431 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:50:58 dist-test-slave-10pc krb5kdc[635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:50:58 dist-test-slave-10pc krb5kdc[635](info): set up 2 sockets
Aug 09 07:50:58 dist-test-slave-10pc krb5kdc[635](info): commencing operation
krb5kdc: starting...
W20260809 07:50:59.979610 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:51:00.112241 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:37781
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:37781,127.18.71.125:33551,127.18.71.124:39315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:00.227811 649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:00.228114 649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:00.228233 649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:00.232255 649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:00.232344 649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:00.232383 649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:00.232420 649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:00.232477 649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:00.238296 649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:37781,127.18.71.125:33551,127.18.71.124:39315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:37781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:00.239858 649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:00.241288 649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:00.252000 654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:00.252007 657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:00.252301 655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:00.252513 649 server_base.cc:1034] running on GCE node
I20260809 07:51:00.253424 649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:00.254942 649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:00.256125 649 hybrid_clock.cc:648] HybridClock initialized: now 1786261860256088 us; error 55 us; skew 500 ppm
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:00.259855 649 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:51:00.261097 649 webserver.cc:460] Webserver started at http://127.18.71.126:45419/ using document root <none> and password file <none>
I20260809 07:51:00.261787 649 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:00.261896 649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:00.262180 649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:00.264907 649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "3a40105accd842c8b846795a806e9256"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "769c7fc4bbfb68714c111c3e89c25a8e"
server_key_iv: "edc5d3a1eed09280158bfb617088ce45"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.265549 649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "3a40105accd842c8b846795a806e9256"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "769c7fc4bbfb68714c111c3e89c25a8e"
server_key_iv: "edc5d3a1eed09280158bfb617088ce45"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.270903 649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 07:51:00.273873 664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:00.275192 649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:51:00.275368 649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "3a40105accd842c8b846795a806e9256"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "769c7fc4bbfb68714c111c3e89c25a8e"
server_key_iv: "edc5d3a1eed09280158bfb617088ce45"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.275492 649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:00.285481 649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:00.288839 649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:00.289055 649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:00.296167 649 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:37781
I20260809 07:51:00.296185 723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:37781 every 8 connection(s)
I20260809 07:51:00.297219 649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:00.299739 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 649
I20260809 07:51:00.299870 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:51:00.300190 18717 external_mini_cluster.cc:1428] Setting key 5cb655ee91d1425b663b3614a3e870a4
I20260809 07:51:00.300350 724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:00.305114 724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.71.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.125@KRBTEST.COM" created.
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:00.321026 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.307989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.18.71.126:37781)
0809 07:51:00.308429 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.308448 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.308768 (+ 320us) server_negotiation.cc:407] Connection header received
0809 07:51:00.309816 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.309821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.310001 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.310202 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.311044 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.311939 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.313057 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.313439 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.313611 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.316121 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.316131 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.316148 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.316179 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.317842 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.318337 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.318343 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.318345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.318451 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.318757 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.318762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.318764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.319139 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.319281 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.319643 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.319942 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":288,"thread_start_us":140,"threads_started":1}
I20260809 07:51:00.321026 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.307990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37781 (local address 127.0.0.1:52822)
0809 07:51:00.308616 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.308660 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.309498 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.310194 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.310199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.310634 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.310893 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.310907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.312079 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.312082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.312931 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.312945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.313599 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.313603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.313724 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.314367 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.314384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.315949 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.317959 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.317969 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.317980 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.318225 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.318544 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.318548 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.318550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.318667 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.319253 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.319259 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.319405 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.319570 (+ 165us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.319992 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"mutex_wait_us":108,"spinlock_wait_cycles":12032,"thread_start_us":163,"threads_started":1}
I20260809 07:51:00.323446 724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:51:00.324172 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.323911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42152)
0809 07:51:00.324032 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.324109 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 07:51:00.324383 665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
I20260809 07:51:00.325241 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.325038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42154)
0809 07:51:00.325141 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.325191 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 07:51:00.325409 724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:33551: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab.
Entry for principal kudu/127.18.71.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab.
I20260809 07:51:00.330937 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:33551
--webserver_interface=127.18.71.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab
--principal=kudu/127.18.71.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:37781,127.18.71.125:33551,127.18.71.124:39315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:00.374680 724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } attempt: 1
I20260809 07:51:00.375769 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.375322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42156)
0809 07:51:00.375544 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.375653 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260809 07:51:00.376724 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.376490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42168)
0809 07:51:00.376602 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.376662 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 07:51:00.376912 724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:33551: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
I20260809 07:51:00.445072 724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } attempt: 2
I20260809 07:51:00.445886 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.445599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42170)
0809 07:51:00.445752 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.445816 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 07:51:00.446610 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.446404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42174)
0809 07:51:00.446507 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.446558 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 07:51:00.446831 724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:33551: Network error: Client connection negotiation failed: client connection to 127.18.71.125:33551: connect: Connection refused (error 111)
W20260809 07:51:00.448408 731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:00.448693 731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:00.448791 731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:00.452776 731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:00.452826 731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:00.452864 731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:00.452903 731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:00.452934 731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:00.458046 731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:37781,127.18.71.125:33551,127.18.71.124:39315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab
--principal=kudu/127.18.71.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:33551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.71.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:00.458992 731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:00.459940 731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:00.468195 737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:00.468214 736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:00.468180 739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:00.468689 731 server_base.cc:1034] running on GCE node
I20260809 07:51:00.469273 731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:00.470305 731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:00.471489 731 hybrid_clock.cc:648] HybridClock initialized: now 1786261860471461 us; error 40 us; skew 500 ppm
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:00.474242 731 init.cc:377] Logged in from keytab as kudu/127.18.71.125@KRBTEST.COM (short username kudu)
I20260809 07:51:00.475064 731 webserver.cc:460] Webserver started at http://127.18.71.125:43731/ using document root <none> and password file <none>
I20260809 07:51:00.475608 731 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:00.475699 731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:00.475929 731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:00.477931 731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data/instance:
uuid: "5ad99e3dd0134f64ae6c47348e7dd740"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "96dbb55b1f7940d5bc1741706cde71c1"
server_key_iv: "ba123ae41bd07e806960b1702f6bfd4a"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.478555 731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/wal/instance:
uuid: "5ad99e3dd0134f64ae6c47348e7dd740"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "96dbb55b1f7940d5bc1741706cde71c1"
server_key_iv: "ba123ae41bd07e806960b1702f6bfd4a"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.482167 731 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 07:51:00.484519 746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:00.485709 731 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:51:00.485845 731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/wal
uuid: "5ad99e3dd0134f64ae6c47348e7dd740"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "96dbb55b1f7940d5bc1741706cde71c1"
server_key_iv: "ba123ae41bd07e806960b1702f6bfd4a"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.485951 731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:00.505381 731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:00.508147 731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:00.508351 731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:00.515318 731 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.125:33551
I20260809 07:51:00.515343 803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.125:33551 every 8 connection(s)
I20260809 07:51:00.516247 731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
I20260809 07:51:00.518198 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 731
I20260809 07:51:00.518332 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-1/wal/instance
I20260809 07:51:00.518664 18717 external_mini_cluster.cc:1428] Setting key bcf19f7135536aff963d6b5a46f45beb
I20260809 07:51:00.519467 804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:00.523564 804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.71.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.124@KRBTEST.COM" created.
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:00.537655 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.525569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.18.71.126:37781)
0809 07:51:00.525706 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.525710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.526714 (+ 1004us) server_negotiation.cc:407] Connection header received
0809 07:51:00.527648 (+ 934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.527652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.527720 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.527826 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.529076 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.529605 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.530964 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.531211 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.531293 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.534019 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.534022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.534024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.534048 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.535538 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.536066 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.536070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.536071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.536164 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.536519 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.536523 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.536524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.536721 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.536833 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.537347 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.537477 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 07:51:00.537998 724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } attempt: 3
I20260809 07:51:00.538573 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.526050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37781 (local address 127.0.0.1:52832)
0809 07:51:00.526565 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.526608 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.527433 (+ 825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.527997 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.528004 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.528336 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.528924 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.528939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.529774 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.529777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.530813 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.530827 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.531372 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.531379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.531628 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.532298 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.532325 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.533840 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.535689 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.535693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.535705 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.535954 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.536259 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.536262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.536264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.536401 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.536833 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.536841 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.537089 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.537269 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.537590 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":158,"threads_started":1}
I20260809 07:51:00.540413 804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab.
Entry for principal kudu/127.18.71.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab.
I20260809 07:51:00.547693 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:39315
--webserver_interface=127.18.71.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab
--principal=kudu/127.18.71.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:37781,127.18.71.125:33551,127.18.71.124:39315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:00.550110 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.540913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42192)
0809 07:51:00.541029 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.541047 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.541126 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.541394 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.541396 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.541560 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.541631 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.541635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.542373 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.542379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.543013 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.543022 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.543533 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.543537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.543660 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.544071 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.544090 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.545807 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.548343 (+ 2536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.548348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.548350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.548613 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.548892 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.548896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.548898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.548979 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.549497 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.549500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.549657 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.549749 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.549935 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":42}
I20260809 07:51:00.550300 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.540940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42192 (local address 127.18.71.125:33551)
0809 07:51:00.541208 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.541215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.541226 (+ 11us) server_negotiation.cc:407] Connection header received
0809 07:51:00.541254 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.541256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.541286 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.541385 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.541735 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.542270 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.543123 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.543397 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.543500 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.545956 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.545963 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.545968 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.545998 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.548241 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.548722 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.548728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.548729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.548805 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.549074 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.549077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.549079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.549393 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.549891 (+ 498us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.549994 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.550120 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":98,"threads_started":1}
I20260809 07:51:00.550410 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.538467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42180)
0809 07:51:00.538641 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.538665 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.538738 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.539626 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.539629 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.539838 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.539959 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.539966 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.541469 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.541472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.542419 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.542428 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.543051 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.543055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.543189 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.543654 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.543669 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.545280 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.548569 (+ 3289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.548575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.548580 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.548859 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.549200 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.549203 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.549205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.549295 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.549782 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.549788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.549911 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.550018 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.550188 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 07:51:00.551128 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.538903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42180 (local address 127.18.71.125:33551)
0809 07:51:00.539202 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.539219 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.539238 (+ 19us) server_negotiation.cc:407] Connection header received
0809 07:51:00.539272 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.539275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.539476 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.539604 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.540074 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.541348 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.542560 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.542926 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.543040 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.545439 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.545450 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.545469 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.545491 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.548399 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.548993 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.548999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.549004 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.549091 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.549407 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.549410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.549411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.549665 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.550792 (+ 1127us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.550875 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.550980 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":212,"thread_start_us":120,"threads_started":1}
I20260809 07:51:00.552630 724 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:51:00.553043 804 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:51:00.553475 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.553170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60084)
0809 07:51:00.553327 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.553395 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 07:51:00.553840 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.553525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60096)
0809 07:51:00.553723 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.553794 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260809 07:51:00.554057 747 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.554363 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.554021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60100)
0809 07:51:00.554218 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.554283 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260809 07:51:00.554497 724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.554844 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.554632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60108)
0809 07:51:00.554739 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.554793 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 07:51:00.555023 804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.585680 724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 1
I20260809 07:51:00.586608 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.586255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60110)
0809 07:51:00.586435 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.586516 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 07:51:00.587381 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.587150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60126)
0809 07:51:00.587263 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.587317 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 07:51:00.587529 724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.604328 804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 1
I20260809 07:51:00.605182 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.604873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60134)
0809 07:51:00.605025 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.605108 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 07:51:00.605929 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.605719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60144)
0809 07:51:00.605825 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.605872 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 07:51:00.606117 804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.662693 724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 2
I20260809 07:51:00.663558 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.663227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60158)
0809 07:51:00.663395 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.663465 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 07:51:00.664249 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.664025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60164)
0809 07:51:00.664128 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.664184 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 07:51:00.664397 724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
W20260809 07:51:00.667328 811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:00.667601 811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:00.667682 811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:00.671753 811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:00.671841 811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:00.671890 811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:00.671923 811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:00.671955 811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:00.674288 804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 2
I20260809 07:51:00.675027 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.674774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60166)
0809 07:51:00.674905 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.674961 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 07:51:00.675740 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.675556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60176)
0809 07:51:00.675650 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.675694 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 07:51:00.675926 804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.677865 811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:37781,127.18.71.125:33551,127.18.71.124:39315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab
--principal=kudu/127.18.71.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:39315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.71.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:00.679015 811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:00.680090 811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:00.688723 818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:00.688768 821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:00.689105 811 server_base.cc:1034] running on GCE node
W20260809 07:51:00.689153 819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:00.689718 811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:00.690876 811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:00.692067 811 hybrid_clock.cc:648] HybridClock initialized: now 1786261860692047 us; error 39 us; skew 500 ppm
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:00.694865 811 init.cc:377] Logged in from keytab as kudu/127.18.71.124@KRBTEST.COM (short username kudu)
I20260809 07:51:00.695786 811 webserver.cc:460] Webserver started at http://127.18.71.124:45729/ using document root <none> and password file <none>
I20260809 07:51:00.696358 811 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:00.696450 811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:00.696755 811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:00.699762 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data/instance:
uuid: "11f2e810f98948dfa115fde2d38cb116"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "276c62cd69af8e2c5cc92a4c2ba0057f"
server_key_iv: "72fda200efbb11e7aa874fdcd01343c2"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.700570 811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/wal/instance:
uuid: "11f2e810f98948dfa115fde2d38cb116"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "276c62cd69af8e2c5cc92a4c2ba0057f"
server_key_iv: "72fda200efbb11e7aa874fdcd01343c2"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.705858 811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 07:51:00.708429 828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:00.709681 811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:51:00.709781 811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/wal
uuid: "11f2e810f98948dfa115fde2d38cb116"
format_stamp: "Formatted at 2026-08-09 07:51:00 on dist-test-slave-10pc"
server_key: "276c62cd69af8e2c5cc92a4c2ba0057f"
server_key_iv: "72fda200efbb11e7aa874fdcd01343c2"
server_key_version: "encryptionkey@0"
I20260809 07:51:00.709864 811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:00.760998 811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:00.763566 724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 3
I20260809 07:51:00.764292 811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:00.764408 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.764068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60182)
0809 07:51:00.764238 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.764304 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 07:51:00.764521 811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:00.765163 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.764910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60192)
0809 07:51:00.765030 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.765090 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 07:51:00.765298 724 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.767094 804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 3
I20260809 07:51:00.767814 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.767538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60204)
0809 07:51:00.767667 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.767717 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 07:51:00.768779 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.768430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60218)
0809 07:51:00.768686 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.768730 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173}
W20260809 07:51:00.768941 804 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:39315: Network error: Client connection negotiation failed: client connection to 127.18.71.124:39315: connect: Connection refused (error 111)
I20260809 07:51:00.772341 811 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.124:39315
I20260809 07:51:00.772351 884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.124:39315 every 8 connection(s)
I20260809 07:51:00.773360 811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
I20260809 07:51:00.775843 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 811
I20260809 07:51:00.775971 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786261695308648-18717-0/minicluster-data/master-2/wal/instance
I20260809 07:51:00.776252 885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:00.776283 18717 external_mini_cluster.cc:1428] Setting key 0d4648e74385a40676e30066018a2f55
I20260809 07:51:00.781386 885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:00.791008 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.777697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.18.71.124:39315)
0809 07:51:00.778103 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.778124 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.778160 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:51:00.779000 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.779020 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.779396 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.779679 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.780664 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.781652 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.782529 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.783257 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.783414 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.786200 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.786205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.786214 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.786239 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.787908 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.788354 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.788359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.788363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.788533 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.788891 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.788893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.788895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.789378 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.789633 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.789920 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.790206 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":136,"threads_started":1}
I20260809 07:51:00.793416 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.783026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52842 (local address 127.18.71.126:37781)
0809 07:51:00.783141 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.783145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.783508 (+ 363us) server_negotiation.cc:407] Connection header received
0809 07:51:00.783773 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.783778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.783835 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.783925 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.784903 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.785390 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.786352 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.786689 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.786801 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.789151 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.789154 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.789156 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.789179 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.790801 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.792093 (+ 1292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.792096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.792097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.792175 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.792501 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.792504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.792505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.792697 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.792825 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.793141 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.793250 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 07:51:00.793435 889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.782964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37781 (local address 127.0.0.1:52842)
0809 07:51:00.783384 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.783434 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.783619 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.784012 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.784016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.784539 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.784756 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.784769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.785521 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.785525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.786220 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.786230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.786802 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.786806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.786940 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.787472 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.787484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.789005 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.790912 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.790922 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.790928 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.791989 (+ 1061us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.792267 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.792270 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.792271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.792368 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.792818 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.792824 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.792932 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.793072 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.793251 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":87,"threads_started":1}
I20260809 07:51:00.794416 885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
I20260809 07:51:00.803212 889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.794855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:33551 (local address 127.0.0.1:42200)
0809 07:51:00.794960 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.794973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.795065 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.795390 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.795393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.795571 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.795682 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.795688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.796517 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.796520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.797249 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.797261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.797730 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.797736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.797879 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.798256 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.798277 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.799496 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.801613 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.801616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.801618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.801888 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.802197 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.802200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.802202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.802305 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.802733 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.802736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.802831 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.802930 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.803067 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 07:51:00.803318 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.793338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52848 (local address 127.18.71.126:37781)
0809 07:51:00.793622 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.793625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.793991 (+ 366us) server_negotiation.cc:407] Connection header received
0809 07:51:00.794107 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.794110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.794159 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.794247 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.794939 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.795484 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.796350 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.796647 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.796723 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.799703 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.799706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.799709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.799736 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.801493 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.801991 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.801994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.801996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.802084 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.802399 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.802402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.802404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.802626 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.802844 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.803011 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.803125 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":93,"threads_started":1}
I20260809 07:51:00.803400 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.794991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42200 (local address 127.18.71.125:33551)
0809 07:51:00.795143 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.795167 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.795179 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:51:00.795222 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.795224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.795279 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.795377 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.795831 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.796369 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.797375 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.797600 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.797688 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.799648 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.799653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.799656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.799684 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.801505 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.801991 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.801994 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.801996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.802084 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.802399 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.802402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.802404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.802626 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.802730 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.803011 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.803125 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 07:51:00.804141 885 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
I20260809 07:51:00.812932 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.804383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42214 (local address 127.18.71.125:33551)
0809 07:51:00.804516 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.804520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.804659 (+ 139us) server_negotiation.cc:407] Connection header received
0809 07:51:00.804854 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.804857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.804894 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.805014 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.805579 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.806032 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.806800 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.807034 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.807146 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.809494 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.809497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.809500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.809523 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.811250 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.811705 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.811708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.811710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.811793 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.812076 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.812079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.812081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.812291 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.812536 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.812671 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.812779 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 07:51:00.812954 889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.804633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60222)
0809 07:51:00.804740 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.804755 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.804823 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.805066 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.805068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.805213 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.805267 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.805271 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.805956 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.805962 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.806443 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.806449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.806908 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.806912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.806983 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.807402 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.807418 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.809311 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.811229 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.811231 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.811232 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.811501 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.811793 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.811796 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.811797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.811907 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.812368 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.812371 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.812484 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.812596 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.812722 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 07:51:00.813037 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.804714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.18.71.124:39315)
0809 07:51:00.804810 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.804813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.804825 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:51:00.804923 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.804926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.804971 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.805055 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.805376 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.805856 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.806570 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.806808 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.806910 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.809455 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.809461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.809463 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.809488 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.811108 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.811595 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.811598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.811599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.811697 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.812002 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.812005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.812006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.812247 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.812353 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.812678 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.812779 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 07:51:00.816754 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:51:00.816771 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116: Bootstrap starting.
I20260809 07:51:00.819321 885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:00.820231 885 log.cc:824] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:00.822284 885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116: No bootstrap required, opened a new log
I20260809 07:51:00.826849 885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:00.827131 885 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:00.827194 885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f2e810f98948dfa115fde2d38cb116, State: Initialized, Role: FOLLOWER
I20260809 07:51:00.827777 885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:00.828867 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.818903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42222 (local address 127.18.71.125:33551)
0809 07:51:00.819010 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.819015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.819505 (+ 490us) server_negotiation.cc:407] Connection header received
0809 07:51:00.819656 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.819659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.819699 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.819747 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.820863 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.821584 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.822886 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.823207 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.823271 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.825067 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.825071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.825073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.825097 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.826607 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.827375 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.827378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.827380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.827472 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.827786 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.827792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.827797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.828000 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.828106 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.828609 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.828722 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 07:51:00.828945 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.819531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.18.71.124:39315)
0809 07:51:00.819671 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.819675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.819687 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:51:00.819718 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.819721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.819755 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.819825 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.820862 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.821536 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.822644 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.822946 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.823034 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.824724 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.824727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.824730 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.824754 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.826530 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.827145 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.827150 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.827152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.827243 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.827599 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.827603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.827604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.827838 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.827962 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.828668 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.828784 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 07:51:00.829277 885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:00.830147 727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.819104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.18.71.126:37781)
0809 07:51:00.819235 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.819238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.819251 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:51:00.819654 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.819657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.819699 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.819747 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.820863 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.821584 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.822886 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.823201 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.823271 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.826491 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.826495 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.826498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.826524 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.828260 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.828765 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.828769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.828770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.828853 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.829181 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.829185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.829187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.829401 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.829506 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.829882 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.829999 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:51:00.831085 900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:00.831280 900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:51:00.834028 911 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:00.834132 911 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:00.832878 811 master_runner.cc:186] Error getting master registration for 127.18.71.126:37781: OK, instance_id { permanent_uuid: "3a40105accd842c8b846795a806e9256" instance_seqno: 1786261860294165 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:00.836324 811 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:00.912098 804 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 4
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:51:00.920909 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.912715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60242 (local address 127.18.71.124:39315)
0809 07:51:00.912844 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.912849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.912906 (+ 57us) server_negotiation.cc:407] Connection header received
0809 07:51:00.913057 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.913060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.913102 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.913199 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.913705 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.914287 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.915030 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.915278 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.915390 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.917729 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.917732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.917735 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.917758 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.919203 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.919647 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.919651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.919653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.919735 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.920022 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.920025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.920027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.920208 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.920312 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.920607 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.920762 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 07:51:00.920909 807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.912655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60242)
0809 07:51:00.912841 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.912857 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.912948 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.913236 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.913239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.913471 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.913586 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.913592 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.914391 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.914396 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.914914 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.914921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.915390 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.915393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.915479 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.916017 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.916028 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.917591 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.919325 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.919328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.919330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.919532 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.919841 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.919844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.919846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.919934 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.920304 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.920307 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.920409 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.920543 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.920728 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 07:51:00.923597 804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: Bootstrap starting.
I20260809 07:51:00.925935 804 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:00.926812 804 log.cc:824] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:00.928870 804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: No bootstrap required, opened a new log
I20260809 07:51:00.929455 724 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } attempt: 4
I20260809 07:51:00.932708 804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:00.932931 804 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:00.932984 804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad99e3dd0134f64ae6c47348e7dd740, State: Initialized, Role: FOLLOWER
I20260809 07:51:00.933455 804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:00.935062 804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:00.935117 914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:00.935216 914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 07:51:00 dist-test-slave-10pc krb5kdc[635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261860, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:51:00.939592 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.929907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:39315 (local address 127.0.0.1:60250)
0809 07:51:00.930087 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:00.930105 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:00.930211 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:00.930466 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:00.930469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:00.930699 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:00.930814 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.930823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.931847 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.931851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.932728 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:00.932739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.933399 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.933403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:00.933492 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.934056 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:00.934089 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:00.935667 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:00.937882 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.937885 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.937887 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.938155 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.938479 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:00.938483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:00.938485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:00.938582 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:00.939079 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:00.939082 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:00.939184 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 07:51:00.939290 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 07:51:00.939431 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 07:51:00.940410 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:00.929964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60250 (local address 127.18.71.124:39315)
0809 07:51:00.930064 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:00.930068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:00.930183 (+ 115us) server_negotiation.cc:407] Connection header received
0809 07:51:00.930300 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:00.930303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:00.930348 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:00.930440 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:00.930960 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.931724 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.932856 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:00.933178 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:00.933281 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:00.935813 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:00.935817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:00.935820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:00.935847 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:00.937748 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.938277 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.938282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.938284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.938372 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:00.938699 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:00.938702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:00.938704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:00.938927 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:00.939029 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:00.940137 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0809 07:51:00.940255 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 07:51:00.940526 926 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:00.940598 926 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:00.938609 731 master_runner.cc:186] Error getting master registration for 127.18.71.126:37781: OK, instance_id { permanent_uuid: "3a40105accd842c8b846795a806e9256" instance_seqno: 1786261860294165 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:51:00.941813 731 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:51:00.943589 724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: Bootstrap starting.
I20260809 07:51:00.946269 724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:00.947151 724 log.cc:824] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:00.949445 724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: No bootstrap required, opened a new log
I20260809 07:51:00.953689 724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:00.953953 724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:00.954015 724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a40105accd842c8b846795a806e9256, State: Initialized, Role: FOLLOWER
I20260809 07:51:00.954478 724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:00.955026 929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:00.955232 929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:00.955850 724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 07:51:00.960999 940 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:51:00.961146 940 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:51:00.966017 649 master_runner.cc:263] No leader master found from master 3a40105accd842c8b846795a806e9256
I20260809 07:51:00.966092 649 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 07:51:01.009999 753 tablet.cc:2369] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:51:01.016214 835 tablet.cc:2369] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:51:01.040756 671 tablet.cc:2369] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:01.297773 900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:01.298002 900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:01.299005 900 leader_election.cc:288] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a40105accd842c8b846795a806e9256 (127.18.71.126:37781), 5ad99e3dd0134f64ae6c47348e7dd740 (127.18.71.125:33551)
I20260809 07:51:01.299548 693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f2e810f98948dfa115fde2d38cb116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a40105accd842c8b846795a806e9256" is_pre_election: true
I20260809 07:51:01.299577 774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f2e810f98948dfa115fde2d38cb116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" is_pre_election: true
I20260809 07:51:01.299841 693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f2e810f98948dfa115fde2d38cb116 in term 0.
I20260809 07:51:01.299896 774 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f2e810f98948dfa115fde2d38cb116 in term 0.
I20260809 07:51:01.300282 831 leader_election.cc:302] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f2e810f98948dfa115fde2d38cb116, 3a40105accd842c8b846795a806e9256; no voters:
I20260809 07:51:01.300597 900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:01.300697 900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:01.300752 900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:01.301834 900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:01.302208 900 leader_election.cc:288] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [CANDIDATE]: Term 1 election: Requested vote from peers 3a40105accd842c8b846795a806e9256 (127.18.71.126:37781), 5ad99e3dd0134f64ae6c47348e7dd740 (127.18.71.125:33551)
I20260809 07:51:01.302501 693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f2e810f98948dfa115fde2d38cb116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a40105accd842c8b846795a806e9256"
I20260809 07:51:01.302549 774 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11f2e810f98948dfa115fde2d38cb116" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ad99e3dd0134f64ae6c47348e7dd740"
I20260809 07:51:01.302635 693 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:01.302695 774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:01.303833 774 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f2e810f98948dfa115fde2d38cb116 in term 1.
I20260809 07:51:01.303839 693 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f2e810f98948dfa115fde2d38cb116 in term 1.
I20260809 07:51:01.304153 829 leader_election.cc:302] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f2e810f98948dfa115fde2d38cb116, 5ad99e3dd0134f64ae6c47348e7dd740; no voters:
I20260809 07:51:01.304327 900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:01.304514 900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [term 1 LEADER]: Becoming Leader. State: Replica: 11f2e810f98948dfa115fde2d38cb116, State: Running, Role: LEADER
I20260809 07:51:01.304828 900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } }
I20260809 07:51:01.307188 942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f2e810f98948dfa115fde2d38cb116. Latest consensus state: current_term: 1 leader_uuid: "11f2e810f98948dfa115fde2d38cb116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:01.307317 942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:01.307678 944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:51:01.308595 944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:51:01.313238 774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [term 1 FOLLOWER]: Refusing update from remote peer 11f2e810f98948dfa115fde2d38cb116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:01.313238 693 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [term 1 FOLLOWER]: Refusing update from remote peer 11f2e810f98948dfa115fde2d38cb116: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:01.313932 900 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:01.314088 942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:01.318264 914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f2e810f98948dfa115fde2d38cb116. Latest consensus state: current_term: 1 leader_uuid: "11f2e810f98948dfa115fde2d38cb116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:01.318264 929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f2e810f98948dfa115fde2d38cb116. Latest consensus state: current_term: 1 leader_uuid: "11f2e810f98948dfa115fde2d38cb116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:01.318400 914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:01.318400 929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:01.320233 900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11f2e810f98948dfa115fde2d38cb116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:01.320351 914 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11f2e810f98948dfa115fde2d38cb116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:01.320492 914 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:01.320833 900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:01.321045 929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11f2e810f98948dfa115fde2d38cb116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:01.321156 929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:51:01.321134 900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11f2e810f98948dfa115fde2d38cb116" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a40105accd842c8b846795a806e9256" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37781 } } peers { permanent_uuid: "5ad99e3dd0134f64ae6c47348e7dd740" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 33551 } } peers { permanent_uuid: "11f2e810f98948dfa115fde2d38cb116" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 39315 } } }
I20260809 07:51:01.321319 900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:01.321350 945 mvcc.cc:205] Tried to move back new op lower bound from 7316528583931977728 to 7316528583907311616. Current Snapshot: MvccSnapshot[applied={T|T < 7316528583931977728}]
I20260809 07:51:01.321429 946 mvcc.cc:205] Tried to move back new op lower bound from 7316528583931977728 to 7316528583907311616. Current Snapshot: MvccSnapshot[applied={T|T < 7316528583931977728}]
I20260809 07:51:01.322661 944 catalog_manager.cc:1347] Generated new cluster ID: 3230be8ad4a24bd3a88330895e7d012d
I20260809 07:51:01.322723 944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:51:01.338804 944 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:51:01.339745 944 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:51:01.347507 944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116: Generated new TSK 0
I20260809 07:51:01.348212 944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:51:01.941999 926 catalog_manager.cc:1259] Loaded cluster ID: 3230be8ad4a24bd3a88330895e7d012d
I20260809 07:51:01.942137 926 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: loading cluster ID for follower catalog manager: success
I20260809 07:51:01.943797 926 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: acquiring CA information for follower catalog manager: success
I20260809 07:51:01.945010 926 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ad99e3dd0134f64ae6c47348e7dd740: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:01.962417 940 catalog_manager.cc:1259] Loaded cluster ID: 3230be8ad4a24bd3a88330895e7d012d
I20260809 07:51:01.962509 940 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: loading cluster ID for follower catalog manager: success
I20260809 07:51:01.964169 940 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: acquiring CA information for follower catalog manager: success
I20260809 07:51:01.965471 940 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3a40105accd842c8b846795a806e9256: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:51:02.422704 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 649
W20260809 07:51:02.432168 831 connection.cc:535] client connection to 127.18.71.126:37781 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 07:51:02.432404 831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:51:02.432494 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 731
I20260809 07:51:02.434952 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:02.434336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37781 (local address 127.0.0.1:52870)
0809 07:51:02.434807 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:02.434868 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:37781: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":221,"threads_started":1}
W20260809 07:51:02.435143 831 consensus_peers.cc:487] T 00000000000000000000000000000000 P 11f2e810f98948dfa115fde2d38cb116 -> Peer 3a40105accd842c8b846795a806e9256 (127.18.71.126:37781): Couldn't send request to peer 3a40105accd842c8b846795a806e9256. Status: Network error: Client connection negotiation failed: client connection to 127.18.71.126:37781: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 07:51:02.440765 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 811
2026-08-09T07:51:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3544 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:51:02 dist-test-slave-10pc krb5kdc[963](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:02 dist-test-slave-10pc krb5kdc[963](info): set up 2 sockets
Aug 09 07:51:02 dist-test-slave-10pc krb5kdc[963](info): commencing operation
krb5kdc: starting...
W20260809 07:51:03.524381 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:51:03 dist-test-slave-10pc krb5kdc[963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:51:03.659497 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33467
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33467
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:03.773264 977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:03.773504 977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:03.773541 977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:03.777535 977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:03.777590 977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:03.777609 977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:03.777626 977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:03.777647 977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:03.782719 977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33467
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:03.783756 977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:03.784873 977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:03.793458 983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:03.793473 982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:03.793464 985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:03.793555 977 server_base.cc:1034] running on GCE node
I20260809 07:51:03.794324 977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:03.795403 977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:03.796594 977 hybrid_clock.cc:648] HybridClock initialized: now 1786261863796553 us; error 42 us; skew 500 ppm
Aug 09 07:51:03 dist-test-slave-10pc krb5kdc[963](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261863, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:03.799980 977 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:51:03.800971 977 webserver.cc:460] Webserver started at http://127.18.71.126:33455/ using document root <none> and password file <none>
I20260809 07:51:03.801537 977 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:03.801610 977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:03.801782 977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:03.803922 977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "46d2d93867654484a10b1732ff842c68"
format_stamp: "Formatted at 2026-08-09 07:51:03 on dist-test-slave-10pc"
server_key: "f31099253f0efe7be13a367426bd8f7b"
server_key_iv: "e9262bf7c7df1a8bbd041e3f3d1244c3"
server_key_version: "encryptionkey@0"
I20260809 07:51:03.804556 977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "46d2d93867654484a10b1732ff842c68"
format_stamp: "Formatted at 2026-08-09 07:51:03 on dist-test-slave-10pc"
server_key: "f31099253f0efe7be13a367426bd8f7b"
server_key_iv: "e9262bf7c7df1a8bbd041e3f3d1244c3"
server_key_version: "encryptionkey@0"
I20260809 07:51:03.808416 977 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:51:03.810995 992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:03.812423 977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:03.812639 977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "46d2d93867654484a10b1732ff842c68"
format_stamp: "Formatted at 2026-08-09 07:51:03 on dist-test-slave-10pc"
server_key: "f31099253f0efe7be13a367426bd8f7b"
server_key_iv: "e9262bf7c7df1a8bbd041e3f3d1244c3"
server_key_version: "encryptionkey@0"
I20260809 07:51:03.812747 977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:03.825021 977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:03.828188 977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:03.828338 977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:03.835413 977 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33467
I20260809 07:51:03.835425 1048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33467 every 8 connection(s)
I20260809 07:51:03.836354 977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:03.836700 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 977
I20260809 07:51:03.836798 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:51:03.837141 18717 external_mini_cluster.cc:1428] Setting key d93ab30f1524d451cb101c5e0c97a551
I20260809 07:51:03.839444 1049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:03.846167 1049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68: Bootstrap starting.
Aug 09 07:51:03 dist-test-slave-10pc krb5kdc[963](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:03.848306 1049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:03.849298 1049 log.cc:824] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:03.851409 1049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68: No bootstrap required, opened a new log
I20260809 07:51:03.852073 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:03.838519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44406 (local address 127.18.71.126:33467)
0809 07:51:03.838937 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:03.838954 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:03.838991 (+ 37us) server_negotiation.cc:407] Connection header received
0809 07:51:03.839707 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:03.839727 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:03.840049 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:03.840326 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:03.841265 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:03.842266 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:03.843522 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:03.843997 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:03.844113 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:03.846956 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:03.846961 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:03.846974 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:03.847000 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:03.849070 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:03.849638 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:03.849644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:03.849650 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:03.849806 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:03.850192 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:03.850194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:03.850196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:03.850582 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:03.850730 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:03.851031 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 07:51:03.851327 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":144,"threads_started":1}
I20260809 07:51:03.855022 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:51:03.854960 1049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2d93867654484a10b1732ff842c68" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33467 } }
I20260809 07:51:03.855229 1049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:03.855281 1049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46d2d93867654484a10b1732ff842c68, State: Initialized, Role: FOLLOWER
I20260809 07:51:03.855751 1049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2d93867654484a10b1732ff842c68" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33467 } }
I20260809 07:51:03.855918 1049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:51:03.855988 1049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:51:03.856061 1049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:03.857033 1049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2d93867654484a10b1732ff842c68" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33467 } }
I20260809 07:51:03.857367 1049 leader_election.cc:302] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46d2d93867654484a10b1732ff842c68; no voters:
I20260809 07:51:03.857687 1049 leader_election.cc:288] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:51:03.857836 1054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:03.858083 1054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [term 1 LEADER]: Becoming Leader. State: Replica: 46d2d93867654484a10b1732ff842c68, State: Running, Role: LEADER
I20260809 07:51:03.858474 1054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2d93867654484a10b1732ff842c68" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33467 } }
I20260809 07:51:03.858881 1049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:03.860034 1055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46d2d93867654484a10b1732ff842c68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2d93867654484a10b1732ff842c68" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33467 } } }
I20260809 07:51:03.860195 1055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:03.860587 1056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46d2d93867654484a10b1732ff842c68. Latest consensus state: current_term: 1 leader_uuid: "46d2d93867654484a10b1732ff842c68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46d2d93867654484a10b1732ff842c68" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33467 } } }
I20260809 07:51:03.860671 1056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:03.860666 1063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:51:03.863466 1063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:51:03.868259 1063 catalog_manager.cc:1347] Generated new cluster ID: 6bd9d298c074489caa8dae47ebdbe7b3
I20260809 07:51:03.868320 1063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:51:03.883829 1063 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:51:03.885092 1063 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:51:03.896054 1063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46d2d93867654484a10b1732ff842c68: Generated new TSK 0
I20260809 07:51:03.896834 1063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:51:03.910809 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 977
2026-08-09T07:51:03Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1468 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:51:03 dist-test-slave-10pc krb5kdc[1077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:03 dist-test-slave-10pc krb5kdc[1077](info): set up 2 sockets
Aug 09 07:51:03 dist-test-slave-10pc krb5kdc[1077](info): commencing operation
krb5kdc: starting...
W20260809 07:51:05.031317 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:51:05 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:51:05.159528 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33631
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:05.272612 1091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:05.272866 1091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:05.272930 1091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:05.276906 1091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:05.276969 1091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:05.276988 1091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:05.277062 1091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:05.277097 1091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:05.282712 1091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:05.283762 1091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:05.284824 1091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:05.293107 1100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:05.293095 1097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:05.293213 1091 server_base.cc:1034] running on GCE node
W20260809 07:51:05.293095 1098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:05.293982 1091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:05.295058 1091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:05.296254 1091 hybrid_clock.cc:648] HybridClock initialized: now 1786261865296225 us; error 40 us; skew 500 ppm
Aug 09 07:51:05 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261865, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:05.299117 1091 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:51:05.300067 1091 webserver.cc:460] Webserver started at http://127.18.71.126:41349/ using document root <none> and password file <none>
I20260809 07:51:05.300691 1091 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:05.300782 1091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:05.301015 1091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:05.303018 1091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "5343d477cc4b41ccb251e1addcffc165"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "ae8415e79202d767188db3d0317bc3ba"
server_key_iv: "f931c906960d7f26f045bed62580ade2"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.303587 1091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "5343d477cc4b41ccb251e1addcffc165"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "ae8415e79202d767188db3d0317bc3ba"
server_key_iv: "f931c906960d7f26f045bed62580ade2"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.307333 1091 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 07:51:05.309723 1108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.310962 1091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:51:05.311101 1091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "5343d477cc4b41ccb251e1addcffc165"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "ae8415e79202d767188db3d0317bc3ba"
server_key_iv: "f931c906960d7f26f045bed62580ade2"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.311221 1091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:05.342376 1091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:05.345533 1091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:05.345755 1091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:05.352850 1091 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33631
I20260809 07:51:05.353129 1161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33631 every 8 connection(s)
I20260809 07:51:05.353893 1091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:05.356859 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 1091
I20260809 07:51:05.356904 1162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:05.357002 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:51:05.357349 18717 external_mini_cluster.cc:1428] Setting key 84ae3fcdb828fd4d32a799fa1b51e990
I20260809 07:51:05.364861 1162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Bootstrap starting.
Aug 09 07:51:05 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:05.367590 1162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:05.368530 1162 log.cc:824] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:05.370527 1162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: No bootstrap required, opened a new log
I20260809 07:51:05.372998 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:05.358741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38832 (local address 127.18.71.126:33631)
0809 07:51:05.359175 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:05.359194 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:05.359239 (+ 45us) server_negotiation.cc:407] Connection header received
0809 07:51:05.359990 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:05.360015 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:05.360480 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:05.360869 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:05.361812 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.363343 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.364231 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.364975 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.365147 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:05.367780 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:05.367785 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:05.367799 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:05.367826 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:05.369786 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:05.370248 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:05.370254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:05.370259 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:05.370387 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:05.370696 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:05.370700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:05.370701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:05.371224 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:05.371408 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:05.371658 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 07:51:05.371950 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260809 07:51:05.374922 1162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:05.375202 1162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:05.375334 1162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5343d477cc4b41ccb251e1addcffc165, State: Initialized, Role: FOLLOWER
I20260809 07:51:05.375771 1162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:05.375927 1162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:51:05.375994 1162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:51:05.376091 1162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:05.377063 1162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:05.377405 1162 leader_election.cc:302] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5343d477cc4b41ccb251e1addcffc165; no voters:
I20260809 07:51:05.377723 1162 leader_election.cc:288] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:51:05.377835 1168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:05.378095 1168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 LEADER]: Becoming Leader. State: Replica: 5343d477cc4b41ccb251e1addcffc165, State: Running, Role: LEADER
I20260809 07:51:05.378427 1168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:05.378979 1162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:05.380090 1169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5343d477cc4b41ccb251e1addcffc165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } } }
I20260809 07:51:05.380349 1170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5343d477cc4b41ccb251e1addcffc165. Latest consensus state: current_term: 1 leader_uuid: "5343d477cc4b41ccb251e1addcffc165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } } }
I20260809 07:51:05.380491 1169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:05.380534 1170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:05.380961 1178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:51:05.383693 1178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:51:05.389122 1178 catalog_manager.cc:1347] Generated new cluster ID: 5feeaeab032540b9a0a32974abd8fb77
I20260809 07:51:05.389186 1178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:51:05.411995 1178 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:51:05.413333 1178 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:51:05.420930 1178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Generated new TSK 0
I20260809 07:51:05.421765 1178 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:51:05.452764 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33631
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:05.563684 1191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:05.563941 1191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:05.563985 1191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:05.567996 1191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:05.568058 1191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:05.568115 1191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:51:05.573496 1191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:05.574534 1191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:05.575558 1191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:05.585414 1196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:05.585433 1197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:05.585420 1199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:05.585613 1191 server_base.cc:1034] running on GCE node
I20260809 07:51:05.586234 1191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:05.586894 1191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:05.588083 1191 hybrid_clock.cc:648] HybridClock initialized: now 1786261865588049 us; error 47 us; skew 500 ppm
Aug 09 07:51:05 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261865, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:05.590963 1191 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:51:05.591858 1191 webserver.cc:460] Webserver started at http://127.18.71.65:34205/ using document root <none> and password file <none>
I20260809 07:51:05.592423 1191 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:05.592571 1191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:05.592819 1191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:05.594832 1191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "ddd29755715042179bed0fb675ceeffe"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "eb629d3a3e135c84c935200fdd2011dc"
server_key_iv: "d7e1be6d15cdac70a8205920fed6e7be"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.595392 1191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "ddd29755715042179bed0fb675ceeffe"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "eb629d3a3e135c84c935200fdd2011dc"
server_key_iv: "d7e1be6d15cdac70a8205920fed6e7be"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.599053 1191 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:51:05.601464 1207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.602664 1191 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:51:05.602797 1191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "ddd29755715042179bed0fb675ceeffe"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "eb629d3a3e135c84c935200fdd2011dc"
server_key_iv: "d7e1be6d15cdac70a8205920fed6e7be"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.602903 1191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:05.625728 1191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:05.628988 1191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:05.629195 1191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:05.629786 1191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:05.630577 1191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:05.630656 1191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.630726 1191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:05.630762 1191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.639307 1191 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:36071
I20260809 07:51:05.639318 1329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:36071 every 8 connection(s)
I20260809 07:51:05.640177 1191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:05.649528 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 1191
I20260809 07:51:05.649683 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:51:05.650038 18717 external_mini_cluster.cc:1428] Setting key c148b710143976aee31f0a25f70a3bf6
Aug 09 07:51:05 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261865, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:05.655037 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:05.642363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:37201 (local address 127.18.71.126:33631)
0809 07:51:05.642870 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:05.642875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:05.643627 (+ 752us) server_negotiation.cc:407] Connection header received
0809 07:51:05.645358 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:05.645361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:05.645409 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:05.645482 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:05.646592 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.647121 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.648210 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.648431 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.648592 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:05.651239 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:05.651243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:05.651246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:05.651271 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:05.652823 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:05.653447 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:05.653450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:05.653451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:05.653527 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:05.653926 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:05.653929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:05.653931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:05.654121 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:05.654234 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:05.654738 (+ 504us) server_negotiation.cc:299] Negotiation successful
0809 07:51:05.654868 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398}
I20260809 07:51:05.655853 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:05.642777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:37201)
0809 07:51:05.643463 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:05.643523 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:05.645148 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:05.645619 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:05.645625 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:05.645980 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:05.646462 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:05.646474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.647231 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.647233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:05.648099 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:05.648107 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.648641 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.648648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:05.648796 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:05.649337 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:05.649353 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:05.651081 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:05.652936 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:05.652945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:05.652964 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:05.653346 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:05.653658 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:05.653665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:05.653669 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:05.653821 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:05.654234 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:05.654242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:05.654493 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 07:51:05.654695 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 07:51:05.655016 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":168,"threads_started":1}
I20260809 07:51:05.657239 1330 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33631
I20260809 07:51:05.657493 1330 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:05.657990 1330 heartbeater.cc:511] Master 127.18.71.126:33631 requested a full tablet report, sending...
I20260809 07:51:05.659569 1126 ts_manager.cc:194] Registered new tserver with Master: ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071)
I20260809 07:51:05.660777 1126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:37201
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:51:05.680150 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33631
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:05.798444 1336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:05.798707 1336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:05.798749 1336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:05.802627 1336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:05.802675 1336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:05.802729 1336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:51:05.807933 1336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:05.808940 1336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:05.809911 1336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:05.819514 1341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:05.819553 1343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:05.819538 1345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:05.819715 1336 server_base.cc:1034] running on GCE node
I20260809 07:51:05.820325 1336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:05.821007 1336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:05.822208 1336 hybrid_clock.cc:648] HybridClock initialized: now 1786261865822193 us; error 25 us; skew 500 ppm
Aug 09 07:51:05 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261865, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:05.824980 1336 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:51:05.825853 1336 webserver.cc:460] Webserver started at http://127.18.71.66:38431/ using document root <none> and password file <none>
I20260809 07:51:05.826488 1336 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:05.826578 1336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:05.826807 1336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:05.828958 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "b140828ef67a4d0baeafb5df465a0d91"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "0825cf508ec43422f2d357e1fabc8be8"
server_key_iv: "424e2c5952b0e09bb971eb970e78e3f5"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.829618 1336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "b140828ef67a4d0baeafb5df465a0d91"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "0825cf508ec43422f2d357e1fabc8be8"
server_key_iv: "424e2c5952b0e09bb971eb970e78e3f5"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.833439 1336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 07:51:05.835768 1352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.837007 1336 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:51:05.837167 1336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b140828ef67a4d0baeafb5df465a0d91"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "0825cf508ec43422f2d357e1fabc8be8"
server_key_iv: "424e2c5952b0e09bb971eb970e78e3f5"
server_key_version: "encryptionkey@0"
I20260809 07:51:05.837296 1336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:05.847501 1336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:05.850322 1336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:05.850520 1336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:05.851090 1336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:05.851909 1336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:05.851982 1336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.852059 1336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:05.852111 1336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:05.861128 1336 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:40937
I20260809 07:51:05.861151 1474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:40937 every 8 connection(s)
I20260809 07:51:05.862047 1336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:05.867200 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 1336
I20260809 07:51:05.867307 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:51:05.867602 18717 external_mini_cluster.cc:1428] Setting key 220fe57aa4ee1e08d8f97dcbd096a1c2
Aug 09 07:51:05 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261865, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:51:05.881774 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:05.863785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:34093 (local address 127.18.71.126:33631)
0809 07:51:05.863927 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:05.863930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:05.864786 (+ 856us) server_negotiation.cc:407] Connection header received
0809 07:51:05.866180 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:05.866183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:05.866232 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:05.866313 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:05.867541 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.868152 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.869757 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.870133 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.870272 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:05.877500 (+ 7228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:05.877502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:05.877504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:05.877530 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:05.879029 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:05.879669 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:05.879672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:05.879674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:05.879778 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:05.880311 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:05.880314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:05.880315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:05.880612 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:05.880814 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:05.881500 (+ 686us) server_negotiation.cc:299] Negotiation successful
0809 07:51:05.881646 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:51:05.882701 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:05.864146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:34093)
0809 07:51:05.864632 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:05.864678 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:05.865958 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:05.866476 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:05.866482 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:05.866835 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:05.867390 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:05.867406 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.868295 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.868297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:05.869631 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:05.869641 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:05.870311 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:05.870314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:05.870461 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:05.875453 (+ 4992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:05.875482 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:05.877298 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:05.879173 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:05.879178 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:05.879191 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:05.879547 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:05.879872 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:05.879874 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:05.879875 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:05.880194 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:05.880824 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:05.880837 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:05.881153 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 07:51:05.881425 (+ 272us) client_negotiation.cc:240] Negotiation successful
0809 07:51:05.881833 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":122,"threads_started":1}
I20260809 07:51:05.884039 1475 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33631
I20260809 07:51:05.884315 1475 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:05.884883 1475 heartbeater.cc:511] Master 127.18.71.126:33631 requested a full tablet report, sending...
I20260809 07:51:05.886006 1126 ts_manager.cc:194] Registered new tserver with Master: b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937)
I20260809 07:51:05.886571 1126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:34093
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:51:05.899324 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33631
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:06.015893 1480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:06.016153 1480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:06.016198 1480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:06.020082 1480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:06.020143 1480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:06.020200 1480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:51:06.025288 1480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:06.026249 1480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:06.027225 1480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:06.036978 1488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.036978 1490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.036974 1485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:06.037429 1480 server_base.cc:1034] running on GCE node
I20260809 07:51:06.037781 1480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:06.038398 1480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:06.039561 1480 hybrid_clock.cc:648] HybridClock initialized: now 1786261866039541 us; error 28 us; skew 500 ppm
Aug 09 07:51:06 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:06.042497 1480 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:51:06.043347 1480 webserver.cc:460] Webserver started at http://127.18.71.67:44789/ using document root <none> and password file <none>
I20260809 07:51:06.043913 1480 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:06.044006 1480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:06.044256 1480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:06.046380 1480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "5100aadf44e14f0fb89c7f68a959b637"
format_stamp: "Formatted at 2026-08-09 07:51:06 on dist-test-slave-10pc"
server_key: "8b8e0420a750b19297ca9c419bc0bf6d"
server_key_iv: "c6ee29459078d7e1b2fd9d2067a16725"
server_key_version: "encryptionkey@0"
I20260809 07:51:06.046960 1480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "5100aadf44e14f0fb89c7f68a959b637"
format_stamp: "Formatted at 2026-08-09 07:51:06 on dist-test-slave-10pc"
server_key: "8b8e0420a750b19297ca9c419bc0bf6d"
server_key_iv: "c6ee29459078d7e1b2fd9d2067a16725"
server_key_version: "encryptionkey@0"
I20260809 07:51:06.050688 1480 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:51:06.053107 1498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.054330 1480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:51:06.054471 1480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "5100aadf44e14f0fb89c7f68a959b637"
format_stamp: "Formatted at 2026-08-09 07:51:06 on dist-test-slave-10pc"
server_key: "8b8e0420a750b19297ca9c419bc0bf6d"
server_key_iv: "c6ee29459078d7e1b2fd9d2067a16725"
server_key_version: "encryptionkey@0"
I20260809 07:51:06.054584 1480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:06.092268 1480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:06.095821 1480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:06.096082 1480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:06.096969 1480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:06.097868 1480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:06.097944 1480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.098028 1480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:06.098062 1480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.109422 1480 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:43033
I20260809 07:51:06.109442 1613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:43033 every 8 connection(s)
I20260809 07:51:06.110328 1480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:06.117326 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 1480
I20260809 07:51:06.117460 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:51:06.117760 18717 external_mini_cluster.cc:1428] Setting key a1a42e0a8d7a9bb8bde0b66bb1ea9547
Aug 09 07:51:06 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:06.124928 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.112142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:52587 (local address 127.18.71.126:33631)
0809 07:51:06.112283 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.112287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.113213 (+ 926us) server_negotiation.cc:407] Connection header received
0809 07:51:06.114690 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.114694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.114754 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.114834 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:06.116038 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.116668 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.118311 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.118747 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.118865 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.121511 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:06.121513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:06.121515 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:06.121540 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:06.123053 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.123488 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.123491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.123492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.123569 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.123926 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.123928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.123929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.124091 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:06.124169 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.124714 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.124812 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 07:51:06.125705 1617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.112579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:52587)
0809 07:51:06.113057 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.113098 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.114417 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.115005 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.115013 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.115364 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:06.115893 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.115908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.116825 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.116831 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.118155 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.118165 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.118915 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.118920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.119103 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.119793 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:06.119809 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:06.121366 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:06.123154 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:06.123158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:06.123169 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:06.123406 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:06.123685 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:06.123688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:06.123689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:06.123835 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:06.124209 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:06.124214 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:06.124428 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.124665 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.124901 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":163,"threads_started":1}
I20260809 07:51:06.126947 1614 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33631
I20260809 07:51:06.127198 1614 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:06.127686 1614 heartbeater.cc:511] Master 127.18.71.126:33631 requested a full tablet report, sending...
I20260809 07:51:06.128722 1126 ts_manager.cc:194] Registered new tserver with Master: 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033)
I20260809 07:51:06.129276 1126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:52587
I20260809 07:51:06.131737 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:51:06.141296 1166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.133736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38840 (local address 127.18.71.126:33631)
0809 07:51:06.133878 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.133882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.134076 (+ 194us) server_negotiation.cc:407] Connection header received
0809 07:51:06.134247 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.134250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.134299 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.134382 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:06.135082 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.135579 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.136437 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.136702 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.136812 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.138197 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:06.138200 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:06.138203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:06.138228 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:06.139450 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.139885 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.139889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.139890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.139978 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.140285 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.140288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.140290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.140607 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:06.140759 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.141028 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.141140 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:51:06.144008 1126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:51:06.147171 1126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:51:06.162362 1630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.156032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:36071 (local address 127.0.0.1:40848)
0809 07:51:06.157111 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.157125 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.157484 (+ 359us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.158286 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.158289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.158321 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:06.158607 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.158619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.160051 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.160053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.160970 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.160980 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.161817 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.161820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.161926 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.161969 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.162082 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.162203 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":82,"spinlock_wait_cycles":4864,"thread_start_us":118,"threads_started":1}
I20260809 07:51:06.162376 1628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.156259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40848 (local address 127.18.71.65:36071)
0809 07:51:06.157799 (+ 1540us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.157815 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.157840 (+ 25us) server_negotiation.cc:407] Connection header received
0809 07:51:06.157921 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.157926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.158122 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.158395 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:06.158774 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.159930 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.161103 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.161688 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.161800 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.161957 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.162131 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.162225 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":67,"threads_started":1}
I20260809 07:51:06.163353 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.156005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40937 (local address 127.0.0.1:41946)
0809 07:51:06.156418 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.156513 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.156649 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.158108 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.158115 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.158137 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:06.158285 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.158293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.159941 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.159945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.161567 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.161572 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.162851 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.162854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.163002 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.163054 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.163118 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.163221 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":822,"thread_start_us":88,"threads_started":1}
I20260809 07:51:06.163507 1632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.157429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43782 (local address 127.18.71.67:43033)
0809 07:51:06.157826 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.157838 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.157859 (+ 21us) server_negotiation.cc:407] Connection header received
0809 07:51:06.157922 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.157926 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.158122 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.158434 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:06.158698 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.159627 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.161708 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.162539 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.162664 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.162817 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.163253 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.163344 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":166,"threads_started":1}
I20260809 07:51:06.163870 1626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.156828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:43033 (local address 127.0.0.1:43782)
0809 07:51:06.157510 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.157526 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.157593 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.158286 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.158289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.158322 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:06.158564 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.158577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.159793 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.159797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.160992 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.161002 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.162849 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.162853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.163008 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.163050 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.163121 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.163730 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"mutex_wait_us":662,"spinlock_wait_cycles":6400,"thread_start_us":114,"threads_started":1}
I20260809 07:51:06.164391 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.156842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.18.71.66:40937)
0809 07:51:06.157222 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.157237 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.157263 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:51:06.157772 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.157777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.157941 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.158118 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:06.158553 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.159783 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.161686 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.162726 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.164029 (+ 1303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.164169 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.164211 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.164275 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":139,"threads_started":1}
I20260809 07:51:06.165841 1259 tablet_service.cc:1467] Processing CreateTablet for tablet 9851ae0592524da9b4ef9d9253be8917 (DEFAULT_TABLE table=test-table [id=bf1affc39a4a4a448c4dc0670de3296a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:06.166455 1547 tablet_service.cc:1467] Processing CreateTablet for tablet 9851ae0592524da9b4ef9d9253be8917 (DEFAULT_TABLE table=test-table [id=bf1affc39a4a4a448c4dc0670de3296a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:06.166818 1259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9851ae0592524da9b4ef9d9253be8917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:06.167536 1547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9851ae0592524da9b4ef9d9253be8917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:06.171243 1403 tablet_service.cc:1467] Processing CreateTablet for tablet 9851ae0592524da9b4ef9d9253be8917 (DEFAULT_TABLE table=test-table [id=bf1affc39a4a4a448c4dc0670de3296a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:06.172284 1403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9851ae0592524da9b4ef9d9253be8917. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:06.173367 1635 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Bootstrap starting.
I20260809 07:51:06.175043 1636 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Bootstrap starting.
I20260809 07:51:06.175879 1635 tablet_bootstrap.cc:654] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:06.176836 1635 log.cc:824] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:06.177306 1636 tablet_bootstrap.cc:654] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:06.178611 1636 log.cc:824] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:06.179086 1639 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Bootstrap starting.
I20260809 07:51:06.179179 1635 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: No bootstrap required, opened a new log
I20260809 07:51:06.179441 1635 ts_tablet_manager.cc:1397] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:51:06.180750 1636 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: No bootstrap required, opened a new log
I20260809 07:51:06.180935 1636 ts_tablet_manager.cc:1397] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:51:06.181607 1639 tablet_bootstrap.cc:654] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:06.182379 1639 log.cc:824] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:06.184007 1639 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: No bootstrap required, opened a new log
I20260809 07:51:06.184217 1639 ts_tablet_manager.cc:1397] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 07:51:06.184231 1635 raft_consensus.cc:348] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.184652 1635 raft_consensus.cc:374] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:06.184757 1635 raft_consensus.cc:729] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd29755715042179bed0fb675ceeffe, State: Initialized, Role: FOLLOWER
I20260809 07:51:06.184839 1636 raft_consensus.cc:348] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.185176 1636 raft_consensus.cc:374] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:06.185245 1636 raft_consensus.cc:729] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5100aadf44e14f0fb89c7f68a959b637, State: Initialized, Role: FOLLOWER
I20260809 07:51:06.185357 1635 consensus_queue.cc:260] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.185720 1636 consensus_queue.cc:260] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.186148 1330 heartbeater.cc:503] Master 127.18.71.126:33631 was elected leader, sending a full tablet report...
I20260809 07:51:06.186496 1635 ts_tablet_manager.cc:1428] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260809 07:51:06.187299 1636 ts_tablet_manager.cc:1428] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:51:06.187673 1614 heartbeater.cc:503] Master 127.18.71.126:33631 was elected leader, sending a full tablet report...
I20260809 07:51:06.187934 1639 raft_consensus.cc:348] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.188184 1639 raft_consensus.cc:374] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:06.188241 1639 raft_consensus.cc:729] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b140828ef67a4d0baeafb5df465a0d91, State: Initialized, Role: FOLLOWER
I20260809 07:51:06.188725 1639 consensus_queue.cc:260] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.189462 1475 heartbeater.cc:503] Master 127.18.71.126:33631 was elected leader, sending a full tablet report...
I20260809 07:51:06.189700 1639 ts_tablet_manager.cc:1428] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 07:51:06.346477 1644 raft_consensus.cc:482] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:06.346767 1644 raft_consensus.cc:504] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.348233 1644 leader_election.cc:288] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071), 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033)
I20260809 07:51:06.353559 1646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.348512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:36071 (local address 127.18.71.66:57123)
0809 07:51:06.349180 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.349192 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.349305 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.349523 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.349529 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.349550 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:06.349647 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.349651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.350586 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.350589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.352178 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.352188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.353129 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.353132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.353233 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.353260 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.353340 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.353408 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":540,"thread_start_us":103,"threads_started":1}
I20260809 07:51:06.353643 1628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.348665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:57123 (local address 127.18.71.65:36071)
0809 07:51:06.348812 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.348817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.349308 (+ 491us) server_negotiation.cc:407] Connection header received
0809 07:51:06.349378 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.349381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.349428 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.349490 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:06.349758 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.350480 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.352315 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.352995 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.353133 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.353188 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.353389 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.353490 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:51:06.353962 1632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.348939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:44087 (local address 127.18.71.67:43033)
0809 07:51:06.349081 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.349084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.349098 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:51:06.349147 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.349149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.349187 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.349257 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:06.349591 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.350420 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.352315 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.352987 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.353140 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.353187 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.353765 (+ 578us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.353842 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:51:06.353684 1477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.348512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:43033 (local address 127.18.71.66:44087)
0809 07:51:06.348668 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.348688 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.348805 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.349295 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.349298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.349311 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:06.349446 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.349451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.350556 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.350562 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.352178 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.352188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.353262 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.353266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.353404 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.353427 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.353502 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.353571 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":167}
I20260809 07:51:06.354465 1281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9851ae0592524da9b4ef9d9253be8917" candidate_uuid: "b140828ef67a4d0baeafb5df465a0d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd29755715042179bed0fb675ceeffe" is_pre_election: true
I20260809 07:51:06.354792 1281 raft_consensus.cc:2454] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b140828ef67a4d0baeafb5df465a0d91 in term 0.
I20260809 07:51:06.354895 1568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9851ae0592524da9b4ef9d9253be8917" candidate_uuid: "b140828ef67a4d0baeafb5df465a0d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5100aadf44e14f0fb89c7f68a959b637" is_pre_election: true
I20260809 07:51:06.355147 1568 raft_consensus.cc:2454] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b140828ef67a4d0baeafb5df465a0d91 in term 0.
I20260809 07:51:06.355337 1355 leader_election.cc:302] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b140828ef67a4d0baeafb5df465a0d91, ddd29755715042179bed0fb675ceeffe; no voters:
I20260809 07:51:06.355654 1644 raft_consensus.cc:2781] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:06.355746 1644 raft_consensus.cc:482] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:06.355824 1644 raft_consensus.cc:3037] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:06.357056 1644 raft_consensus.cc:504] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.357465 1644 leader_election.cc:288] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [CANDIDATE]: Term 1 election: Requested vote from peers ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071), 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033)
I20260809 07:51:06.357801 1281 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9851ae0592524da9b4ef9d9253be8917" candidate_uuid: "b140828ef67a4d0baeafb5df465a0d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd29755715042179bed0fb675ceeffe"
I20260809 07:51:06.357844 1568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9851ae0592524da9b4ef9d9253be8917" candidate_uuid: "b140828ef67a4d0baeafb5df465a0d91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5100aadf44e14f0fb89c7f68a959b637"
I20260809 07:51:06.357954 1281 raft_consensus.cc:3037] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:06.357959 1568 raft_consensus.cc:3037] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:06.359318 1568 raft_consensus.cc:2454] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b140828ef67a4d0baeafb5df465a0d91 in term 1.
I20260809 07:51:06.359318 1281 raft_consensus.cc:2454] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b140828ef67a4d0baeafb5df465a0d91 in term 1.
I20260809 07:51:06.359642 1354 leader_election.cc:302] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5100aadf44e14f0fb89c7f68a959b637, b140828ef67a4d0baeafb5df465a0d91; no voters:
I20260809 07:51:06.359835 1644 raft_consensus.cc:2781] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:06.360146 1644 raft_consensus.cc:686] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 1 LEADER]: Becoming Leader. State: Replica: b140828ef67a4d0baeafb5df465a0d91, State: Running, Role: LEADER
I20260809 07:51:06.360553 1644 consensus_queue.cc:237] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
W20260809 07:51:06.361313 1615 tablet.cc:2369] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:51:06.363128 1476 tablet.cc:2369] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:06.363947 1125 catalog_manager.cc:5495] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 reported cstate change: term changed from 0 to 1, leader changed from <none> to b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "b140828ef67a4d0baeafb5df465a0d91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } health_report { overall_health: UNKNOWN } } }
W20260809 07:51:06.391534 1331 tablet.cc:2369] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:06.396746 1631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.393375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.18.71.66:40937)
0809 07:51:06.393497 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.393500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.393513 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:51:06.393588 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.393590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.393644 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.393738 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:51:06.393970 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.394462 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.395300 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.395529 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.395634 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.395888 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:51:06.396319 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:51:06.396414 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.396552 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.396611 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:51:06.402628 1568 raft_consensus.cc:1261] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 1 FOLLOWER]: Refusing update from remote peer b140828ef67a4d0baeafb5df465a0d91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:06.402628 1281 raft_consensus.cc:1261] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Refusing update from remote peer b140828ef67a4d0baeafb5df465a0d91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:06.403481 1647 consensus_queue.cc:1035] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:06.403684 1644 consensus_queue.cc:1035] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:06.412496 1651 mvcc.cc:205] Tried to move back new op lower bound from 7316528604777558016 to 7316528604616187904. Current Snapshot: MvccSnapshot[applied={T|T < 7316528604777558016}]
I20260809 07:51:06.424510 1125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:06.424697 1125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:06.427207 1125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Sending DeleteTablet for 3 replicas of tablet 9851ae0592524da9b4ef9d9253be8917
I20260809 07:51:06.427986 1403 tablet_service.cc:1514] Processing DeleteTablet for tablet 9851ae0592524da9b4ef9d9253be8917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:06 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:41946
I20260809 07:51:06.427986 1259 tablet_service.cc:1514] Processing DeleteTablet for tablet 9851ae0592524da9b4ef9d9253be8917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:06 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40848
I20260809 07:51:06.428299 1547 tablet_service.cc:1514] Processing DeleteTablet for tablet 9851ae0592524da9b4ef9d9253be8917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:06 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:43782
I20260809 07:51:06.428474 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 1191
I20260809 07:51:06.428503 1660 tablet_replica.cc:330] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: stopping tablet replica
I20260809 07:51:06.434765 1661 tablet_replica.cc:330] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: stopping tablet replica
I20260809 07:51:06.435109 1661 raft_consensus.cc:2229] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:06.435453 1661 raft_consensus.cc:2258] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:06.437319 1660 raft_consensus.cc:2229] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:51:06.437745 1660 raft_consensus.cc:2258] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:06.438594 1661 ts_tablet_manager.cc:1905] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:06.439225 1660 ts_tablet_manager.cc:1905] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:06.442018 1660 ts_tablet_manager.cc:1918] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:06.442205 1660 log.cc:1196] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/9851ae0592524da9b4ef9d9253be8917
I20260809 07:51:06.442286 1661 ts_tablet_manager.cc:1918] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:06.442493 1661 log.cc:1196] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/9851ae0592524da9b4ef9d9253be8917
I20260809 07:51:06.442598 1660 ts_tablet_manager.cc:1939] T 9851ae0592524da9b4ef9d9253be8917 P b140828ef67a4d0baeafb5df465a0d91: Deleting consensus metadata
I20260809 07:51:06.442934 1661 ts_tablet_manager.cc:1939] T 9851ae0592524da9b4ef9d9253be8917 P 5100aadf44e14f0fb89c7f68a959b637: Deleting consensus metadata
I20260809 07:51:06.443851 1109 catalog_manager.cc:4841] TS b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937): tablet 9851ae0592524da9b4ef9d9253be8917 (table test-table [id=bf1affc39a4a4a448c4dc0670de3296a]) successfully deleted
I20260809 07:51:06.444205 1110 catalog_manager.cc:4841] TS 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033): tablet 9851ae0592524da9b4ef9d9253be8917 (table test-table [id=bf1affc39a4a4a448c4dc0670de3296a]) successfully deleted
W20260809 07:51:06.445307 1111 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:36071 (error 108)
I20260809 07:51:06.446138 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 1336
I20260809 07:51:06.446389 1626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.446080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:36071 (local address 127.0.0.1:37956)
0809 07:51:06.446216 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.446292 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:36071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 07:51:06.448686 1111 catalog_manager.cc:4584] TS ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9851ae0592524da9b4ef9d9253be8917: Network error: Client connection negotiation failed: client connection to 127.18.71.65:36071: connect: Connection refused (error 111)
I20260809 07:51:06.458909 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 1480
I20260809 07:51:06.466804 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 1091
I20260809 07:51:06.474565 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33631
--webserver_interface=127.18.71.126
--webserver_port=41349
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:06.588099 1663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:06.588393 1663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:06.588436 1663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:06.592618 1663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:06.592705 1663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:06.592724 1663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:06.592742 1663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:06.592758 1663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:06.598233 1663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=41349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:06.599370 1663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:06.600472 1663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:06.614843 1673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:06.614948 1663 server_base.cc:1034] running on GCE node
W20260809 07:51:06.614840 1670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.614833 1671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:06.615864 1663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:06.617067 1663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:06.618235 1663 hybrid_clock.cc:648] HybridClock initialized: now 1786261866618213 us; error 34 us; skew 500 ppm
Aug 09 07:51:06 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:06.621052 1663 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:51:06.621953 1663 webserver.cc:460] Webserver started at http://127.18.71.126:41349/ using document root <none> and password file <none>
I20260809 07:51:06.622555 1663 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:06.622682 1663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:06.626551 1663 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 07:51:06.628713 1680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.629936 1663 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:06.630090 1663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "5343d477cc4b41ccb251e1addcffc165"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "ae8415e79202d767188db3d0317bc3ba"
server_key_iv: "f931c906960d7f26f045bed62580ade2"
server_key_version: "encryptionkey@0"
I20260809 07:51:06.630741 1663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:06.641378 1663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:06.644212 1663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:06.644428 1663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:06.652628 1663 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33631
I20260809 07:51:06.652931 1733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33631 every 8 connection(s)
I20260809 07:51:06.653816 1663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:06.658317 1734 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:06.659915 1734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Bootstrap starting.
I20260809 07:51:06.661378 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 1663
I20260809 07:51:06.661891 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:36071
--local_ip_for_outbound_sockets=127.18.71.65
--tserver_master_addrs=127.18.71.126:33631
--webserver_port=34205
--webserver_interface=127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:06.678162 1734 log.cc:824] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:06.687337 1734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:06.687739 1734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Bootstrap complete.
I20260809 07:51:06.691905 1734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:06.692299 1734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5343d477cc4b41ccb251e1addcffc165, State: Initialized, Role: FOLLOWER
I20260809 07:51:06.692816 1734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:06.692934 1734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:51:06.693022 1734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:51:06.693145 1734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:06.695032 1734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:06.695364 1734 leader_election.cc:302] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5343d477cc4b41ccb251e1addcffc165; no voters:
I20260809 07:51:06.695711 1734 leader_election.cc:288] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 07:51:06.695794 1738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:51:06.695987 1738 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 LEADER]: Becoming Leader. State: Replica: 5343d477cc4b41ccb251e1addcffc165, State: Running, Role: LEADER
I20260809 07:51:06.696285 1738 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:06.697001 1734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:06.697300 1738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5343d477cc4b41ccb251e1addcffc165. Latest consensus state: current_term: 2 leader_uuid: "5343d477cc4b41ccb251e1addcffc165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } } }
I20260809 07:51:06.697425 1738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:06.697844 1739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5343d477cc4b41ccb251e1addcffc165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } } }
I20260809 07:51:06.698017 1739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:06.699062 1742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:51:06.703320 1742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:51:06.703846 1742 catalog_manager.cc:1259] Loaded cluster ID: 5feeaeab032540b9a0a32974abd8fb77
I20260809 07:51:06.703920 1742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:51:06.705063 1742 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 07:51:06.707774 1742 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 07:51:06.711956 1742 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:51:06.712711 1742 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Loaded TSK: 0
I20260809 07:51:06.713678 1742 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 07:51:06.790861 1736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:06.791168 1736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:06.791256 1736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:06.795372 1736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:06.795467 1736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:06.795562 1736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:51:06.801330 1736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:36071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=34205
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:06.802322 1736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:06.803311 1736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:06.812965 1763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.813004 1762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:06.813031 1765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:06.813544 1736 server_base.cc:1034] running on GCE node
I20260809 07:51:06.814000 1736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:06.814778 1736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:06.815989 1736 hybrid_clock.cc:648] HybridClock initialized: now 1786261866815947 us; error 50 us; skew 500 ppm
Aug 09 07:51:06 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:06.819451 1736 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:51:06.820605 1736 webserver.cc:460] Webserver started at http://127.18.71.65:34205/ using document root <none> and password file <none>
I20260809 07:51:06.821321 1736 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:06.821439 1736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:06.826640 1736 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 07:51:06.829480 1773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:06.831039 1736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 07:51:06.831177 1736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "ddd29755715042179bed0fb675ceeffe"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "eb629d3a3e135c84c935200fdd2011dc"
server_key_iv: "d7e1be6d15cdac70a8205920fed6e7be"
server_key_version: "encryptionkey@0"
I20260809 07:51:06.831895 1736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:06.851361 1736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:06.854931 1736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:06.855121 1736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:06.855792 1736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:06.857227 1782 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 07:51:06.861714 1736 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 07:51:06.861790 1736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 07:51:06.861852 1736 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 07:51:06.863960 1736 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 07:51:06.864017 1736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 07:51:06.864352 1782 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Bootstrap starting.
I20260809 07:51:06.874352 1736 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:36071
I20260809 07:51:06.874478 1895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:36071 every 8 connection(s)
I20260809 07:51:06.875355 1736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:06.879869 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 1736
I20260809 07:51:06.880375 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:40937
--local_ip_for_outbound_sockets=127.18.71.66
--tserver_master_addrs=127.18.71.126:33631
--webserver_port=38431
--webserver_interface=127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:06.885735 1782 log.cc:824] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:51:06 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:06.893898 1782 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:06.894358 1782 tablet_bootstrap.cc:492] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Bootstrap complete.
I20260809 07:51:06.895812 1782 ts_tablet_manager.cc:1397] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.007s
I20260809 07:51:06.896021 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.877559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:33337)
0809 07:51:06.878270 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.878327 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.879587 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.881094 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.881106 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.881638 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:06.882378 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.882410 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.884255 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.884259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.885566 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.885577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.886445 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.886449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.886677 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.887422 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:06.887450 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:06.889484 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:06.892296 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:06.892301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:06.892314 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:06.892680 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:06.893158 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:06.893161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:06.893163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:06.893332 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:06.894149 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:06.894158 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:06.894462 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.894658 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.894999 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":157,"threads_started":1}
I20260809 07:51:06.896054 1898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.877581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:33337 (local address 127.18.71.126:33631)
0809 07:51:06.878017 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.878037 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.878463 (+ 426us) server_negotiation.cc:407] Connection header received
0809 07:51:06.880108 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.880124 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.880579 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.880948 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:06.882609 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.884089 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.885691 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.886282 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.886459 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.889719 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:06.889727 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:06.889746 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:06.889788 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:06.892159 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.892808 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.892818 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.892843 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.893015 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.893459 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.893465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.893468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.893995 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:06.894167 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.894848 (+ 681us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.895178 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":153,"threads_started":1}
I20260809 07:51:06.898766 1896 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33631
I20260809 07:51:06.898980 1896 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:06.899490 1896 heartbeater.cc:511] Master 127.18.71.126:33631 requested a full tablet report, sending...
I20260809 07:51:06.899277 1782 raft_consensus.cc:348] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.899816 1782 raft_consensus.cc:729] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd29755715042179bed0fb675ceeffe, State: Initialized, Role: FOLLOWER
I20260809 07:51:06.900300 1782 consensus_queue.cc:260] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } } peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } }
I20260809 07:51:06.902361 1782 ts_tablet_manager.cc:1428] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260809 07:51:06.902823 1697 ts_manager.cc:194] Registered new tserver with Master: ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071)
I20260809 07:51:06.906345 1697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:33337
I20260809 07:51:06.908896 1896 heartbeater.cc:503] Master 127.18.71.126:33631 was elected leader, sending a full tablet report...
Aug 09 07:51:06 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261866, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:51:06.916677 1905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.905572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37964 (local address 127.18.71.65:36071)
0809 07:51:06.907096 (+ 1524us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:06.907107 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:06.907129 (+ 22us) server_negotiation.cc:407] Connection header received
0809 07:51:06.907199 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:06.907204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:06.907397 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:06.907533 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:06.908249 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.909375 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.910268 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.910720 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.910835 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.913101 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:06.913109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:06.913118 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:06.913139 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:06.914729 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.915294 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.915298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.915300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.915370 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:06.915677 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:06.915679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:06.915680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:06.915870 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:06.915962 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:06.916338 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 07:51:06.916507 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1418,"thread_start_us":90,"threads_started":1}
I20260809 07:51:06.916706 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:06.905472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:36071 (local address 127.0.0.1:37964)
0809 07:51:06.905870 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:06.905918 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:06.906039 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:06.907566 (+ 1527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:06.907571 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:06.907989 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:06.908113 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.908121 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.909489 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.909492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.910139 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:06.910147 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:06.910840 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:06.910844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:06.910957 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:06.911494 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:06.911506 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:06.912949 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:06.914874 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:06.914883 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:06.914888 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:06.915187 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:06.915461 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:06.915467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:06.915471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:06.915580 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:06.915979 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:06.915984 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:06.916104 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 07:51:06.916270 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 07:51:06.916531 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
I20260809 07:51:06.917851 1828 tablet_service.cc:1514] Processing DeleteTablet for tablet 9851ae0592524da9b4ef9d9253be8917 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:06 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:37964
I20260809 07:51:06.918635 1906 tablet_replica.cc:330] stopping tablet replica
I20260809 07:51:06.918918 1906 raft_consensus.cc:2229] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:06.919163 1906 raft_consensus.cc:2258] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:06.920063 1906 ts_tablet_manager.cc:1905] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:06.923434 1906 ts_tablet_manager.cc:1918] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:06.923545 1906 log.cc:1196] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/9851ae0592524da9b4ef9d9253be8917
I20260809 07:51:06.923810 1906 ts_tablet_manager.cc:1939] T 9851ae0592524da9b4ef9d9253be8917 P ddd29755715042179bed0fb675ceeffe: Deleting consensus metadata
I20260809 07:51:06.925423 1683 catalog_manager.cc:4841] TS ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071): tablet 9851ae0592524da9b4ef9d9253be8917 (table test-table [id=bf1affc39a4a4a448c4dc0670de3296a]) successfully deleted
W20260809 07:51:07.005339 1900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:07.005599 1900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:07.005674 1900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:07.009644 1900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:07.009699 1900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:07.009797 1900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:51:07.014933 1900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:40937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=38431
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:07.016000 1900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:07.017072 1900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:07.026646 1915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:07.026662 1912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.026646 1900 server_base.cc:1034] running on GCE node
W20260809 07:51:07.026679 1911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.027233 1900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:07.027869 1900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:07.029052 1900 hybrid_clock.cc:648] HybridClock initialized: now 1786261867029037 us; error 30 us; skew 500 ppm
Aug 09 07:51:07 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:07.031837 1900 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:51:07.032835 1900 webserver.cc:460] Webserver started at http://127.18.71.66:38431/ using document root <none> and password file <none>
I20260809 07:51:07.033483 1900 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:07.033560 1900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:07.037539 1900 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 07:51:07.039604 1922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.040861 1900 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 07:51:07.040979 1900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b140828ef67a4d0baeafb5df465a0d91"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "0825cf508ec43422f2d357e1fabc8be8"
server_key_iv: "424e2c5952b0e09bb971eb970e78e3f5"
server_key_version: "encryptionkey@0"
I20260809 07:51:07.041531 1900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:07.056237 1900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:07.059495 1900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:07.059688 1900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:07.060263 1900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:07.061080 1900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:07.061128 1900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.061200 1900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:07.061234 1900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.070032 1900 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:40937
I20260809 07:51:07.070072 2051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:40937 every 8 connection(s)
I20260809 07:51:07.070984 1900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:07.079663 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 1900
I20260809 07:51:07.080017 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:43033
--local_ip_for_outbound_sockets=127.18.71.67
--tserver_master_addrs=127.18.71.126:33631
--webserver_port=44789
--webserver_interface=127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:51:07 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:07.086445 1898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.072792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:58165 (local address 127.18.71.126:33631)
0809 07:51:07.072946 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.072950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.073745 (+ 795us) server_negotiation.cc:407] Connection header received
0809 07:51:07.075188 (+ 1443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.075192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.075258 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.075364 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:07.076650 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.077197 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.078342 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.078581 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.078700 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.082242 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:07.082245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:07.082247 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:07.082270 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:07.083916 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:07.084509 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:07.084515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:07.084520 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:07.084614 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:07.084981 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:07.084987 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:07.084992 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:07.085287 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:07.085397 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.086027 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.086196 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:51:07.087229 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.073150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:58165)
0809 07:51:07.073589 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:07.073630 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:07.074967 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:07.075547 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:07.075554 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:07.075907 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:07.076501 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.076526 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.077325 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.077329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.078215 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.078224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.078747 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.078750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.078918 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.080348 (+ 1430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:07.080365 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:07.082081 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:07.084056 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:07.084061 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:07.084074 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:07.084354 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:07.084718 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:07.084721 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:07.084723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:07.084867 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:07.085410 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:07.085416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:07.085762 (+ 346us) client_negotiation.cc:769] Sending connection context
0809 07:51:07.085938 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 07:51:07.086227 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":125,"threads_started":1}
I20260809 07:51:07.088603 2052 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33631
I20260809 07:51:07.088882 2052 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:07.089548 2052 heartbeater.cc:511] Master 127.18.71.126:33631 requested a full tablet report, sending...
I20260809 07:51:07.090793 1697 ts_manager.cc:194] Registered new tserver with Master: b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937)
I20260809 07:51:07.091391 1697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:58165
W20260809 07:51:07.197978 2055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:07.198275 2055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:07.198361 2055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:07.202473 2055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:07.202528 2055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:07.202620 2055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:51:07.207912 2055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:43033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=44789
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:07.209086 2055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:07.210247 2055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:07.220438 2061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:07.220466 2060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.220676 2055 server_base.cc:1034] running on GCE node
W20260809 07:51:07.220496 2063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.221290 2055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:07.221942 2055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:07.223109 2055 hybrid_clock.cc:648] HybridClock initialized: now 1786261867223096 us; error 32 us; skew 500 ppm
Aug 09 07:51:07 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:07.226023 2055 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:51:07.226894 2055 webserver.cc:460] Webserver started at http://127.18.71.67:44789/ using document root <none> and password file <none>
I20260809 07:51:07.227419 2055 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:07.227492 2055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:07.231244 2055 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:51:07.233494 2070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.234835 2055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 07:51:07.234967 2055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "5100aadf44e14f0fb89c7f68a959b637"
format_stamp: "Formatted at 2026-08-09 07:51:06 on dist-test-slave-10pc"
server_key: "8b8e0420a750b19297ca9c419bc0bf6d"
server_key_iv: "c6ee29459078d7e1b2fd9d2067a16725"
server_key_version: "encryptionkey@0"
I20260809 07:51:07.235563 2055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:07.249794 2055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:07.253260 2055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:07.253472 2055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:07.254074 2055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:07.254918 2055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:07.254990 2055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.255064 2055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:07.255117 2055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.264305 2055 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:43033
I20260809 07:51:07.264330 2191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:43033 every 8 connection(s)
I20260809 07:51:07.265460 2055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:07.268774 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 2055
Aug 09 07:51:07 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:07.278416 2197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.269432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.18.71.126:33631)
0809 07:51:07.269680 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.269684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.269946 (+ 262us) server_negotiation.cc:407] Connection header received
0809 07:51:07.270048 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.270051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.270105 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.270201 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:07.270992 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.271735 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.272808 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.273157 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.273264 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.274893 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:07.274896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:07.274899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:07.274920 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:07.276687 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:07.277143 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:07.277145 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:07.277146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:07.277209 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:07.277507 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:07.277510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:07.277512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:07.277690 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:07.277914 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.278043 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.278237 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260809 07:51:07.280761 1898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.267448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:37289 (local address 127.18.71.126:33631)
0809 07:51:07.267593 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.267597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.268414 (+ 817us) server_negotiation.cc:407] Connection header received
0809 07:51:07.270083 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.270087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.270136 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.270228 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:07.271553 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.272208 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.273657 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.273954 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.274033 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.276853 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:07.276855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:07.276857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:07.276875 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:07.278451 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:07.279072 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:07.279076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:07.279077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:07.279160 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:07.279585 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:07.279588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:07.279590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:07.279787 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:07.279882 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.280447 (+ 565us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.280586 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":58}
I20260809 07:51:07.281561 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.267818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:37289)
0809 07:51:07.268266 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:07.268309 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:07.269837 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:07.270416 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:07.270423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:07.270774 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:07.271406 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.271424 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.272333 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.272335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.273545 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.273555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.274119 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.274126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.274363 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.274961 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:07.274978 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:07.276693 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:07.278569 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:07.278577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:07.278594 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:07.278988 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:07.279258 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:07.279261 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:07.279263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:07.279464 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:07.279891 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:07.279895 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:07.280155 (+ 260us) client_negotiation.cc:769] Sending connection context
0809 07:51:07.280389 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 07:51:07.280744 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
I20260809 07:51:07.282614 2192 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33631
I20260809 07:51:07.282841 2192 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:07.283294 2192 heartbeater.cc:511] Master 127.18.71.126:33631 requested a full tablet report, sending...
I20260809 07:51:07.284250 1697 ts_manager.cc:194] Registered new tserver with Master: 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033)
I20260809 07:51:07.284839 1697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:37289
I20260809 07:51:07.292155 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:51:07.296208 1898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.292844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.18.71.126:33631)
0809 07:51:07.292985 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.292991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.293160 (+ 169us) server_negotiation.cc:407] Connection header received
0809 07:51:07.293296 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.293298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.293390 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.293475 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:51:07.293676 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.294172 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.294960 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.295166 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.295237 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.295490 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:51:07.295854 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:51:07.295942 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.295992 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.296033 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:51:07.296861 1697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:51:07.300393 1697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:51:07.308373 1828 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa (DEFAULT_TABLE table=test-table [id=a5235defd5ac416ebaf0d68a9b6b9e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:07.309072 1828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:07.312153 1782 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Bootstrap starting.
I20260809 07:51:07.313109 1904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.307228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:43033 (local address 127.0.0.1:33760)
0809 07:51:07.307401 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:07.307415 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:07.307493 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:07.308386 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:07.308392 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:07.308416 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:07.308543 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.308552 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.310008 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.310011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.311296 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.311307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.312650 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.312654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.312751 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.312794 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 07:51:07.312879 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 07:51:07.312975 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":49}
I20260809 07:51:07.313159 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.307400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33760 (local address 127.18.71.67:43033)
0809 07:51:07.307849 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.307862 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.307888 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:51:07.307972 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.307978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.308214 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.308397 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:07.308698 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.309843 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.311434 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.312512 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.312652 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.312821 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.312911 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.313017 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":176,"threads_started":1}
I20260809 07:51:07.313423 1782 tablet_bootstrap.cc:654] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:07.314054 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.307707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40937 (local address 127.0.0.1:58246)
0809 07:51:07.307963 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:07.307978 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:07.308064 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:07.309248 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:07.309251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:07.309269 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:07.309366 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.309373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.310648 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.310654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.311693 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.311701 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.313563 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.313575 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.313698 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.313728 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 07:51:07.313817 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 07:51:07.313907 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":2,"thread_start_us":105,"threads_started":1}
I20260809 07:51:07.314556 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.308144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.18.71.66:40937)
0809 07:51:07.308656 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.308667 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.308689 (+ 22us) server_negotiation.cc:407] Connection header received
0809 07:51:07.308771 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.308775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.308995 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.309176 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:07.309499 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.310511 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.311814 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.312851 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.312955 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.313088 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.314290 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.314405 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":158,"threads_started":1}
I20260809 07:51:07.315393 1782 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: No bootstrap required, opened a new log
I20260809 07:51:07.315495 1782 ts_tablet_manager.cc:1397] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:07.315896 1782 raft_consensus.cc:348] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.316028 1782 raft_consensus.cc:374] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:07.316080 1782 raft_consensus.cc:729] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd29755715042179bed0fb675ceeffe, State: Initialized, Role: FOLLOWER
I20260809 07:51:07.316258 1782 consensus_queue.cc:260] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.316607 1782 ts_tablet_manager.cc:1428] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:51:07.316867 2121 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa (DEFAULT_TABLE table=test-table [id=a5235defd5ac416ebaf0d68a9b6b9e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:07.317459 1976 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa (DEFAULT_TABLE table=test-table [id=a5235defd5ac416ebaf0d68a9b6b9e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:07.318343 2121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:07.318461 1976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:07.326853 2208 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Bootstrap starting.
I20260809 07:51:07.327205 2209 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Bootstrap starting.
I20260809 07:51:07.329702 2209 tablet_bootstrap.cc:654] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:07.330188 2208 tablet_bootstrap.cc:654] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:07.331120 2209 log.cc:824] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:07.331120 2208 log.cc:824] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:07.333303 2208 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: No bootstrap required, opened a new log
I20260809 07:51:07.333410 2209 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: No bootstrap required, opened a new log
I20260809 07:51:07.333523 2208 ts_tablet_manager.cc:1397] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:51:07.333626 2209 ts_tablet_manager.cc:1397] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:51:07.336758 2209 raft_consensus.cc:348] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.336747 2208 raft_consensus.cc:348] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.336994 2209 raft_consensus.cc:374] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:07.336990 2208 raft_consensus.cc:374] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:07.337064 2208 raft_consensus.cc:729] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b140828ef67a4d0baeafb5df465a0d91, State: Initialized, Role: FOLLOWER
I20260809 07:51:07.337065 2209 raft_consensus.cc:729] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5100aadf44e14f0fb89c7f68a959b637, State: Initialized, Role: FOLLOWER
I20260809 07:51:07.337517 2208 consensus_queue.cc:260] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.337517 2209 consensus_queue.cc:260] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.338199 2052 heartbeater.cc:503] Master 127.18.71.126:33631 was elected leader, sending a full tablet report...
I20260809 07:51:07.338207 2192 heartbeater.cc:503] Master 127.18.71.126:33631 was elected leader, sending a full tablet report...
I20260809 07:51:07.338565 2209 ts_tablet_manager.cc:1428] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 07:51:07.338567 2208 ts_tablet_manager.cc:1428] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260809 07:51:07.376894 1897 tablet.cc:2369] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:51:07.516713 2195 tablet.cc:2369] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:07.527318 2213 raft_consensus.cc:482] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:07.527525 2213 raft_consensus.cc:504] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.528753 2213 leader_election.cc:288] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937), ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071)
I20260809 07:51:07.534016 2206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.529162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:50361 (local address 127.18.71.66:40937)
0809 07:51:07.529298 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.529303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.529616 (+ 313us) server_negotiation.cc:407] Connection header received
0809 07:51:07.529679 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.529685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.529747 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.529819 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:07.530058 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.530865 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.532670 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.533304 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.533434 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.533508 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.533810 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.533881 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:51:07.534019 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.529142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:44639 (local address 127.18.71.65:36071)
0809 07:51:07.529455 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.529458 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.529472 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:51:07.529522 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.529524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.529607 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.529743 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:07.530058 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.530865 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.532651 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.533334 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.533440 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.533618 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.533810 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.533882 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":144,"threads_started":1}
I20260809 07:51:07.534046 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.529030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:36071 (local address 127.18.71.67:44639)
0809 07:51:07.529249 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:07.529265 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:07.529371 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:07.529757 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:07.529760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:07.529772 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:07.529916 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.529925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.530979 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.530983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.532519 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.532530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.533460 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.533464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.533645 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.533702 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 07:51:07.533776 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 07:51:07.533883 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":188}
I20260809 07:51:07.534111 2214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.529030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40937 (local address 127.18.71.67:50361)
0809 07:51:07.529518 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:07.529528 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:07.529604 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:07.529837 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:07.529840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:07.529851 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:07.529947 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.529956 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.530979 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.530983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.532519 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:07.532530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.533460 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.533464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:07.533617 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.533698 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 07:51:07.533776 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 07:51:07.533883 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":249,"thread_start_us":113,"threads_started":1}
I20260809 07:51:07.534852 2001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1d7340e84e4ec08978ce3cdea8b8aa" candidate_uuid: "5100aadf44e14f0fb89c7f68a959b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b140828ef67a4d0baeafb5df465a0d91" is_pre_election: true
I20260809 07:51:07.534931 1848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1d7340e84e4ec08978ce3cdea8b8aa" candidate_uuid: "5100aadf44e14f0fb89c7f68a959b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd29755715042179bed0fb675ceeffe" is_pre_election: true
I20260809 07:51:07.535195 2001 raft_consensus.cc:2454] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5100aadf44e14f0fb89c7f68a959b637 in term 0.
I20260809 07:51:07.535254 1848 raft_consensus.cc:2454] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5100aadf44e14f0fb89c7f68a959b637 in term 0.
I20260809 07:51:07.535705 2071 leader_election.cc:302] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5100aadf44e14f0fb89c7f68a959b637, b140828ef67a4d0baeafb5df465a0d91, ddd29755715042179bed0fb675ceeffe; no voters:
I20260809 07:51:07.535988 2213 raft_consensus.cc:2781] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:07.536093 2213 raft_consensus.cc:482] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:07.536154 2213 raft_consensus.cc:3037] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:07.537400 2213 raft_consensus.cc:504] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.537729 2213 leader_election.cc:288] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [CANDIDATE]: Term 1 election: Requested vote from peers b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937), ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071)
I20260809 07:51:07.538070 1848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1d7340e84e4ec08978ce3cdea8b8aa" candidate_uuid: "5100aadf44e14f0fb89c7f68a959b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddd29755715042179bed0fb675ceeffe"
I20260809 07:51:07.538216 1848 raft_consensus.cc:3037] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:07.539243 1848 raft_consensus.cc:2454] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5100aadf44e14f0fb89c7f68a959b637 in term 1.
I20260809 07:51:07.539515 2001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1d7340e84e4ec08978ce3cdea8b8aa" candidate_uuid: "5100aadf44e14f0fb89c7f68a959b637" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b140828ef67a4d0baeafb5df465a0d91"
I20260809 07:51:07.539575 2073 leader_election.cc:302] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5100aadf44e14f0fb89c7f68a959b637, ddd29755715042179bed0fb675ceeffe; no voters:
I20260809 07:51:07.539640 2001 raft_consensus.cc:3037] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:07.539758 2213 raft_consensus.cc:2781] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:07.540014 2213 raft_consensus.cc:686] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 1 LEADER]: Becoming Leader. State: Replica: 5100aadf44e14f0fb89c7f68a959b637, State: Running, Role: LEADER
I20260809 07:51:07.540390 2213 consensus_queue.cc:237] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:07.540956 2001 raft_consensus.cc:2454] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5100aadf44e14f0fb89c7f68a959b637 in term 1.
I20260809 07:51:07.543882 1697 catalog_manager.cc:5495] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "5100aadf44e14f0fb89c7f68a959b637" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } health_report { overall_health: UNKNOWN } } }
W20260809 07:51:07.572083 2053 tablet.cc:2369] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:07.608675 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.605366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33776 (local address 127.18.71.67:43033)
0809 07:51:07.605489 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:07.605493 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:07.605506 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:51:07.605643 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:07.605645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:07.605709 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:07.605801 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:51:07.606086 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.606566 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.607349 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:07.607550 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:07.607622 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:07.607882 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:51:07.608284 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:51:07.608373 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:07.608435 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 07:51:07.608538 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:51:07.614866 2001 raft_consensus.cc:1261] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 1 FOLLOWER]: Refusing update from remote peer 5100aadf44e14f0fb89c7f68a959b637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:07.614862 1848 raft_consensus.cc:1261] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Refusing update from remote peer 5100aadf44e14f0fb89c7f68a959b637: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:07.615640 2216 consensus_queue.cc:1035] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:07.615826 2213 consensus_queue.cc:1035] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:07.621851 2219 mvcc.cc:205] Tried to move back new op lower bound from 7316528609742864384 to 7316528609448370176. Current Snapshot: MvccSnapshot[applied={T|T < 7316528609742864384}]
I20260809 07:51:07.632396 1697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:07.632606 1697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:07.635070 1697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Sending DeleteTablet for 3 replicas of tablet 0a1d7340e84e4ec08978ce3cdea8b8aa
I20260809 07:51:07.635779 2121 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:07 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:33760
I20260809 07:51:07.635790 1976 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:07 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:58246
I20260809 07:51:07.635977 1828 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:07 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:37964
I20260809 07:51:07.636310 2230 tablet_replica.cc:330] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: stopping tablet replica
I20260809 07:51:07.636473 2230 raft_consensus.cc:2229] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:07.636610 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 1736
I20260809 07:51:07.636613 2230 raft_consensus.cc:2258] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:07.636613 2229 tablet_replica.cc:330] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: stopping tablet replica
I20260809 07:51:07.638971 2229 raft_consensus.cc:2229] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:07.639544 2229 raft_consensus.cc:2258] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:07.641211 2229 ts_tablet_manager.cc:1905] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:07.644225 2229 ts_tablet_manager.cc:1918] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:07.644383 2229 log.cc:1196] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/0a1d7340e84e4ec08978ce3cdea8b8aa
I20260809 07:51:07.644829 2229 ts_tablet_manager.cc:1939] T 0a1d7340e84e4ec08978ce3cdea8b8aa P b140828ef67a4d0baeafb5df465a0d91: Deleting consensus metadata
I20260809 07:51:07.644973 2228 tablet_replica.cc:330] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: stopping tablet replica
I20260809 07:51:07.645321 2228 raft_consensus.cc:2229] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:51:07.645769 2228 raft_consensus.cc:2258] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:07.646415 1681 catalog_manager.cc:4841] TS b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937): tablet 0a1d7340e84e4ec08978ce3cdea8b8aa (table test-table [id=a5235defd5ac416ebaf0d68a9b6b9e90]) successfully deleted
W20260809 07:51:07.647212 1683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:36071 (error 108)
I20260809 07:51:07.647471 2228 ts_tablet_manager.cc:1905] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:07.647569 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 1900
I20260809 07:51:07.649147 2204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:07.648160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:36071 (local address 127.0.0.1:37972)
0809 07:51:07.648888 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:07.649026 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:36071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":603}
W20260809 07:51:07.649362 1683 catalog_manager.cc:4584] TS ddd29755715042179bed0fb675ceeffe (127.18.71.65:36071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a1d7340e84e4ec08978ce3cdea8b8aa: Network error: Client connection negotiation failed: client connection to 127.18.71.65:36071: connect: Connection refused (error 111)
I20260809 07:51:07.651422 2228 ts_tablet_manager.cc:1918] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:07.651528 2228 log.cc:1196] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/0a1d7340e84e4ec08978ce3cdea8b8aa
I20260809 07:51:07.651965 2228 ts_tablet_manager.cc:1939] T 0a1d7340e84e4ec08978ce3cdea8b8aa P 5100aadf44e14f0fb89c7f68a959b637: Deleting consensus metadata
I20260809 07:51:07.653445 1682 catalog_manager.cc:4841] TS 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033): tablet 0a1d7340e84e4ec08978ce3cdea8b8aa (table test-table [id=a5235defd5ac416ebaf0d68a9b6b9e90]) successfully deleted
I20260809 07:51:07.658267 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 2055
I20260809 07:51:07.666972 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 1663
I20260809 07:51:07.674825 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33631
--webserver_interface=127.18.71.126
--webserver_port=41349
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33631
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:07.788385 2232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:07.788664 2232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:07.788709 2232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:07.792681 2232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:07.792744 2232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:07.792764 2232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:07.792831 2232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:07.792861 2232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:07.798225 2232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=41349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:07.799314 2232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:07.800365 2232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:07.809008 2240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:07.809026 2238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.809247 2232 server_base.cc:1034] running on GCE node
W20260809 07:51:07.809008 2237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:07.809945 2232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:07.811048 2232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:07.812239 2232 hybrid_clock.cc:648] HybridClock initialized: now 1786261867812225 us; error 31 us; skew 500 ppm
Aug 09 07:51:07 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261867, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:07.815157 2232 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:51:07.816006 2232 webserver.cc:460] Webserver started at http://127.18.71.126:41349/ using document root <none> and password file <none>
I20260809 07:51:07.816638 2232 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:07.816735 2232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:07.820437 2232 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.002s
I20260809 07:51:07.822643 2247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:07.823823 2232 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:51:07.823961 2232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "5343d477cc4b41ccb251e1addcffc165"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "ae8415e79202d767188db3d0317bc3ba"
server_key_iv: "f931c906960d7f26f045bed62580ade2"
server_key_version: "encryptionkey@0"
I20260809 07:51:07.824564 2232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:07.846426 2232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:07.849344 2232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:07.849553 2232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:07.856400 2232 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33631
I20260809 07:51:07.856768 2302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33631 every 8 connection(s)
I20260809 07:51:07.857574 2232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:07.861781 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 2232
I20260809 07:51:07.862087 2303 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:07.862286 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:36071
--local_ip_for_outbound_sockets=127.18.71.65
--tserver_master_addrs=127.18.71.126:33631
--webserver_port=34205
--webserver_interface=127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:07.864652 2303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Bootstrap starting.
I20260809 07:51:07.882668 2303 log.cc:824] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:07.894212 2303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:07.894624 2303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165: Bootstrap complete.
I20260809 07:51:07.898939 2303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:07.899339 2303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5343d477cc4b41ccb251e1addcffc165, State: Initialized, Role: FOLLOWER
I20260809 07:51:07.899837 2303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:07.899957 2303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:51:07.900050 2303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:51:07.900177 2303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:51:07.902158 2303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:07.902504 2303 leader_election.cc:302] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5343d477cc4b41ccb251e1addcffc165; no voters:
I20260809 07:51:07.902884 2303 leader_election.cc:288] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 07:51:07.902969 2307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:51:07.903196 2307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [term 3 LEADER]: Becoming Leader. State: Replica: 5343d477cc4b41ccb251e1addcffc165, State: Running, Role: LEADER
I20260809 07:51:07.903506 2307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } }
I20260809 07:51:07.904206 2303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:07.904493 2307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5343d477cc4b41ccb251e1addcffc165. Latest consensus state: current_term: 3 leader_uuid: "5343d477cc4b41ccb251e1addcffc165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } } }
I20260809 07:51:07.904613 2307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:07.905023 2311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:51:07.905030 2308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5343d477cc4b41ccb251e1addcffc165" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5343d477cc4b41ccb251e1addcffc165" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33631 } } }
I20260809 07:51:07.905304 2308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5343d477cc4b41ccb251e1addcffc165 [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:07.910555 2311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:51:07.911300 2311 catalog_manager.cc:1259] Loaded cluster ID: 5feeaeab032540b9a0a32974abd8fb77
I20260809 07:51:07.911499 2311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 07:51:07.912569 2311 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6f257a99ad google::LogMessage::Fail() at ??:0
@ 0x7f6f257ac88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6f257a94de google::LogMessage::Flush() at ??:0
@ 0x7f6f257aaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6f300ecfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6f300edd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6f300ea579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6f3012e640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6f279fb3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6f279fbc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6f279fd3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6f279ec512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6f290306db start_thread at ??:0
@ 0x7f6f242a761f clone at ??:0
W20260809 07:51:08.007143 2305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:08.007381 2305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:08.007423 2305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:08.011480 2305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:08.011538 2305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:08.011592 2305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:51:08.016912 2305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:36071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=34205
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:08.017975 2305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:08.019019 2305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:51:08.029002 2305 server_base.cc:1034] running on GCE node
W20260809 07:51:08.029199 2350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:08.029245 2351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:08.029389 2353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:08.029824 2305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:08.030485 2305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:08.031694 2305 hybrid_clock.cc:648] HybridClock initialized: now 1786261868031664 us; error 49 us; skew 500 ppm
Aug 09 07:51:08 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261868, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:08.034960 2305 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:51:08.035923 2305 webserver.cc:460] Webserver started at http://127.18.71.65:34205/ using document root <none> and password file <none>
I20260809 07:51:08.036624 2305 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:08.036728 2305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:08.040884 2305 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:08.043221 2361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.044662 2305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:51:08.044813 2305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "ddd29755715042179bed0fb675ceeffe"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "eb629d3a3e135c84c935200fdd2011dc"
server_key_iv: "d7e1be6d15cdac70a8205920fed6e7be"
server_key_version: "encryptionkey@0"
I20260809 07:51:08.045413 2305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:08.057519 2305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:08.060815 2305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:08.061025 2305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:08.061602 2305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:08.062722 2368 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 07:51:08.068430 2305 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 07:51:08.068548 2305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 07:51:08.068630 2305 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 07:51:08.070272 2305 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 07:51:08.070340 2305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 07:51:08.070673 2368 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Bootstrap starting.
I20260809 07:51:08.081256 2305 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:36071
I20260809 07:51:08.082194 2305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:08.082755 2482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:36071 every 8 connection(s)
I20260809 07:51:08.090912 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 2305
I20260809 07:51:08.091388 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:40937
--local_ip_for_outbound_sockets=127.18.71.66
--tserver_master_addrs=127.18.71.126:33631
--webserver_port=38431
--webserver_interface=127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:08.093951 2368 log.cc:824] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:08.101189 2368 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:08.101574 2368 tablet_bootstrap.cc:492] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Bootstrap complete.
I20260809 07:51:08.102787 2368 ts_tablet_manager.cc:1397] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Time spent bootstrapping tablet: real 0.032s user 0.025s sys 0.004s
I20260809 07:51:08.105391 2368 raft_consensus.cc:348] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:08.105841 2368 raft_consensus.cc:729] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd29755715042179bed0fb675ceeffe, State: Initialized, Role: FOLLOWER
I20260809 07:51:08.106324 2368 consensus_queue.cc:260] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:08.107335 2368 ts_tablet_manager.cc:1428] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
W20260809 07:51:08.127818 2485 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:51:08.084262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:40279)
0809 07:51:08.084746 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.084799 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:08.085633 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:08.127020 (+ 41387us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":154,"threads_started":1}
W20260809 07:51:08.128513 2483 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:33631 (0 consecutive failures): Network error: Failed to ping master at 127.18.71.126:33631: Client connection negotiation failed: client connection to 127.18.71.126:33631: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:51:08.129249 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.129000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:35973)
0809 07:51:08.129123 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.129197 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 07:51:08.129911 2485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.129709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:48503)
0809 07:51:08.129812 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.129856 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 07:51:08.130118 2483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 07:51:08.217999 2486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:08.218307 2486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:08.218386 2486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:08.222530 2486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:08.222615 2486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:08.222712 2486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:51:08.227878 2486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:40937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=38431
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:08.228969 2486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:08.229976 2486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:08.240036 2497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:08.240192 2494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:08.240036 2493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:08.240525 2486 server_base.cc:1034] running on GCE node
I20260809 07:51:08.241241 2486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:08.242188 2486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:08.243381 2486 hybrid_clock.cc:648] HybridClock initialized: now 1786261868243357 us; error 35 us; skew 500 ppm
Aug 09 07:51:08 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261868, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:08.247151 2486 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:51:08.248240 2486 webserver.cc:460] Webserver started at http://127.18.71.66:38431/ using document root <none> and password file <none>
I20260809 07:51:08.248878 2486 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:08.248945 2486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:08.252890 2486 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:08.255021 2504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.256271 2486 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:08.256407 2486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b140828ef67a4d0baeafb5df465a0d91"
format_stamp: "Formatted at 2026-08-09 07:51:05 on dist-test-slave-10pc"
server_key: "0825cf508ec43422f2d357e1fabc8be8"
server_key_iv: "424e2c5952b0e09bb971eb970e78e3f5"
server_key_version: "encryptionkey@0"
I20260809 07:51:08.257054 2486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:08.286688 2486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:08.290458 2486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:08.290685 2486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:08.291317 2486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:08.292236 2486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:08.292311 2486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.292379 2486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:08.292428 2486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.301535 2486 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:40937
I20260809 07:51:08.301568 2629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:40937 every 8 connection(s)
I20260809 07:51:08.302548 2486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:08.306541 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.304623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:41625)
0809 07:51:08.305140 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.305269 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":127,"threads_started":1}
W20260809 07:51:08.306985 2630 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:33631 (0 consecutive failures): Network error: Failed to ping master at 127.18.71.126:33631: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
I20260809 07:51:08.307595 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.307373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:46391)
0809 07:51:08.307494 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.307542 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 07:51:08.308156 2632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.307977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:39321)
0809 07:51:08.308071 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.308110 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 07:51:08.308336 2630 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 07:51:08.310499 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 2486
I20260809 07:51:08.310999 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:43033
--local_ip_for_outbound_sockets=127.18.71.67
--tserver_master_addrs=127.18.71.126:33631
--webserver_port=44789
--webserver_interface=127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:33817
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:08.426239 2633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:08.426482 2633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:08.426625 2633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:08.430661 2633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:08.430718 2633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:08.430832 2633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:51:08.436136 2633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:33817
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:43033
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=44789
--tserver_master_addrs=127.18.71.126:33631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:08.437157 2633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:08.438184 2633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:51:08.447881 2633 server_base.cc:1034] running on GCE node
W20260809 07:51:08.448091 2640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:08.448117 2639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:08.448105 2642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:08.448743 2633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:08.449404 2633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:08.450564 2633 hybrid_clock.cc:648] HybridClock initialized: now 1786261868450547 us; error 29 us; skew 500 ppm
Aug 09 07:51:08 dist-test-slave-10pc krb5kdc[1077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261868, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:08.453452 2633 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:51:08.454360 2633 webserver.cc:460] Webserver started at http://127.18.71.67:44789/ using document root <none> and password file <none>
I20260809 07:51:08.454901 2633 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:08.454964 2633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:08.458655 2633 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:51:08.460721 2649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.461962 2633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:08.462080 2633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "5100aadf44e14f0fb89c7f68a959b637"
format_stamp: "Formatted at 2026-08-09 07:51:06 on dist-test-slave-10pc"
server_key: "8b8e0420a750b19297ca9c419bc0bf6d"
server_key_iv: "c6ee29459078d7e1b2fd9d2067a16725"
server_key_version: "encryptionkey@0"
I20260809 07:51:08.462603 2633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:08.478576 2633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:08.481827 2633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:08.482038 2633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:08.482648 2633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:08.483485 2633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:08.483533 2633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.483601 2633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:08.483633 2633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:08.493103 2633 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:43033
I20260809 07:51:08.493129 2775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:43033 every 8 connection(s)
I20260809 07:51:08.494115 2633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:08.497952 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.496235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:46833)
0809 07:51:08.496723 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.496831 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":180,"threads_started":1}
I20260809 07:51:08.498323 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 2633
W20260809 07:51:08.498422 2776 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:33631 (0 consecutive failures): Network error: Failed to ping master at 127.18.71.126:33631: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
I20260809 07:51:08.499033 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.498828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:50573)
0809 07:51:08.498939 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.498984 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 07:51:08.499713 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:08.499500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:56977)
0809 07:51:08.499629 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:08.499667 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260809 07:51:08.499897 2776 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 07:51:09.131108 2783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:09.130665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:43029)
0809 07:51:09.130924 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:09.130980 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":76,"threads_started":1}
I20260809 07:51:09.309422 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:09.308969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:36585)
0809 07:51:09.309238 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:09.309311 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260809 07:51:09.500808 2785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:09.500392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:42283)
0809 07:51:09.500661 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:09.500711 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":117,"threads_started":1}
I20260809 07:51:09.707706 2786 raft_consensus.cc:482] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:09.707990 2786 raft_consensus.cc:504] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:09.709290 2786 leader_election.cc:288] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033), b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937)
Aug 09 07:51:09 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261868, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.67@KRBTEST.COM
Aug 09 07:51:09 dist-test-slave-10pc krb5kdc[1077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261868, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.66@KRBTEST.COM
I20260809 07:51:09.731022 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:09.709465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:43033 (local address 127.18.71.65:54173)
0809 07:51:09.709868 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:09.709883 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:09.710024 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:09.712266 (+ 2242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:09.712269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:09.712821 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:09.713663 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:09.713687 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.715646 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.715659 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:09.717317 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:09.717329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.718341 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.718345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:09.718627 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:09.723257 (+ 4630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:09.723282 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:09.725199 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:09.727769 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:09.727780 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:09.727797 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:09.728285 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:09.728716 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:09.728719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:09.728722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:09.728938 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:09.729792 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:09.729800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:09.730172 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 07:51:09.730390 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 07:51:09.730854 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":590,"spinlock_wait_cycles":29440,"thread_start_us":96,"threads_started":1}
I20260809 07:51:09.731045 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:09.709686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:54173 (local address 127.18.71.67:43033)
0809 07:51:09.710196 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:09.710216 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:09.710250 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:51:09.711193 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:09.711202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:09.711676 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:09.712031 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:09.713870 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.715470 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.717446 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.718206 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.718384 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:09.725447 (+ 7063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:09.725452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:09.725468 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:09.725496 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:09.727624 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:09.728423 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:09.728430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:09.728436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:09.728605 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:09.729053 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:09.729056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:09.729058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:09.729654 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:09.729796 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:09.730553 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 07:51:09.730851 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":131,"threads_started":1}
I20260809 07:51:09.731045 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:09.709853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:54247 (local address 127.18.71.66:40937)
0809 07:51:09.710157 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:09.710187 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:09.710240 (+ 53us) server_negotiation.cc:407] Connection header received
0809 07:51:09.711193 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:09.711202 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:09.711677 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:09.712031 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:09.713870 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.715470 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.717446 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.718206 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.718384 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:09.725753 (+ 7369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:09.725761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:09.725779 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:09.725818 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:09.727702 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:09.728423 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:09.728430 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:09.728436 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:09.728605 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:09.729053 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:09.729056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:09.729058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:09.729662 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:09.729796 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:09.730553 (+ 757us) server_negotiation.cc:299] Negotiation successful
0809 07:51:09.730851 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":59,"threads_started":1}
I20260809 07:51:09.731022 2787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:09.709542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:40937 (local address 127.18.71.65:54247)
0809 07:51:09.709894 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:09.709908 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:09.710023 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:09.712188 (+ 2165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:09.712195 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:09.712821 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:09.713663 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:09.713687 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.715646 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.715651 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:09.717318 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:09.717329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:09.718342 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:09.718345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:09.718627 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:09.723275 (+ 4648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:09.723295 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:09.725532 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:09.727799 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:09.727801 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:09.727804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:09.728285 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:09.728716 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:09.728719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:09.728722 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:09.728938 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:09.729792 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:09.729800 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:09.730172 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 07:51:09.730390 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 07:51:09.730803 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":391,"spinlock_wait_cycles":97024,"thread_start_us":61,"threads_started":1}
W20260809 07:51:09.734247 2362 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
W20260809 07:51:09.734448 2363 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
I20260809 07:51:09.734551 2363 leader_election.cc:302] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd29755715042179bed0fb675ceeffe; no voters: 5100aadf44e14f0fb89c7f68a959b637, b140828ef67a4d0baeafb5df465a0d91
I20260809 07:51:09.734838 2786 raft_consensus.cc:2726] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:51:10.132258 2788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:10.131929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:34263)
0809 07:51:10.132101 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:10.132165 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 07:51:10.310770 2791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:10.310349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:58849)
0809 07:51:10.310598 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:10.310669 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
I20260809 07:51:10.501965 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:10.501546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:41687)
0809 07:51:10.501811 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:10.501872 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":72,"threads_started":1}
I20260809 07:51:11.133524 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:11.133018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:41665)
0809 07:51:11.133304 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:11.133371 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":98,"threads_started":1}
I20260809 07:51:11.311946 2794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:11.311500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:49893)
0809 07:51:11.311779 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:11.311846 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":82,"threads_started":1}
I20260809 07:51:11.503141 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:11.502714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:58917)
0809 07:51:11.502984 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:11.503040 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":76,"threads_started":1}
I20260809 07:51:11.728292 2796 raft_consensus.cc:482] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:11.728459 2796 raft_consensus.cc:504] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:11.728919 2796 leader_election.cc:288] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033), b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937)
W20260809 07:51:11.729857 2363 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
W20260809 07:51:11.729969 2362 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
I20260809 07:51:11.730118 2362 leader_election.cc:302] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd29755715042179bed0fb675ceeffe; no voters: 5100aadf44e14f0fb89c7f68a959b637, b140828ef67a4d0baeafb5df465a0d91
I20260809 07:51:11.730311 2796 raft_consensus.cc:2726] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:51:12.134725 2797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:12.134260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:51133)
0809 07:51:12.134527 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:12.134586 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":75,"threads_started":1}
I20260809 07:51:12.313233 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:12.312822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:52769)
0809 07:51:12.313082 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:12.313142 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260809 07:51:12.504207 2799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:12.503834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:40653)
0809 07:51:12.504067 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:12.504118 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":74,"threads_started":1}
I20260809 07:51:13.136137 2800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:13.135646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:58995)
0809 07:51:13.135974 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:13.136035 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":75,"threads_started":1}
I20260809 07:51:13.314399 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:13.313986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:49825)
0809 07:51:13.314242 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:13.314302 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260809 07:51:13.505411 2803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:13.505030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:57687)
0809 07:51:13.505270 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:13.505322 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
I20260809 07:51:14.137367 2804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:14.136960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:60755)
0809 07:51:14.137220 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:14.137272 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":90,"threads_started":1}
I20260809 07:51:14.315627 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:14.315182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:60301)
0809 07:51:14.315458 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:14.315528 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":116,"threads_started":1}
I20260809 07:51:14.506678 2807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:14.506240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:36335)
0809 07:51:14.506520 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:14.506573 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260809 07:51:15.051532 2808 raft_consensus.cc:482] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:15.051672 2808 raft_consensus.cc:504] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:15.052179 2808 leader_election.cc:288] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033), b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937)
W20260809 07:51:15.053088 2362 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
W20260809 07:51:15.053190 2363 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
I20260809 07:51:15.053236 2362 leader_election.cc:302] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd29755715042179bed0fb675ceeffe; no voters: 5100aadf44e14f0fb89c7f68a959b637, b140828ef67a4d0baeafb5df465a0d91
I20260809 07:51:15.053395 2808 raft_consensus.cc:2726] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:51:15.138617 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:15.138181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:39229)
0809 07:51:15.138461 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:15.138518 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260809 07:51:15.317042 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:15.316568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:35063)
0809 07:51:15.316889 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:15.316945 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":91,"threads_started":1}
I20260809 07:51:15.507943 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:15.507498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:60601)
0809 07:51:15.507793 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:15.507851 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":73,"threads_started":1}
I20260809 07:51:16.139900 2813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:16.139417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:49625)
0809 07:51:16.139702 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:16.139778 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":82,"threads_started":1}
I20260809 07:51:16.318168 2814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:16.317724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:46527)
0809 07:51:16.318000 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:16.318050 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":97,"threads_started":1}
I20260809 07:51:16.509428 2815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:16.508906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:40499)
0809 07:51:16.509216 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:16.509278 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":117,"threads_started":1}
I20260809 07:51:17.141155 2816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:17.140724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:50303)
0809 07:51:17.140999 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:17.141057 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":81,"threads_started":1}
I20260809 07:51:17.319515 2817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:17.319087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:37233)
0809 07:51:17.319355 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:17.319415 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":78,"threads_started":1}
I20260809 07:51:17.510615 2818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:17.510180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:42199)
0809 07:51:17.510464 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:17.510522 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":100,"threads_started":1}
I20260809 07:51:18.142544 2819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:18.142096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:42233)
0809 07:51:18.142378 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:18.142443 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":100,"threads_started":1}
I20260809 07:51:18.320783 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:18.320316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:38943)
0809 07:51:18.320616 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:18.320675 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":80,"threads_started":1}
I20260809 07:51:18.511827 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:18.511404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:45545)
0809 07:51:18.511680 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:18.511737 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":88,"threads_started":1}
I20260809 07:51:19.143862 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:19.143427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:42517)
0809 07:51:19.143710 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:19.143766 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260809 07:51:19.322134 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:19.321563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:34845)
0809 07:51:19.321974 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:19.322035 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":103,"threads_started":1}
I20260809 07:51:19.513101 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:19.512696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:53503)
0809 07:51:19.512956 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:19.513010 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":98,"threads_started":1}
I20260809 07:51:20.094533 2825 raft_consensus.cc:482] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:20.094681 2825 raft_consensus.cc:504] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5100aadf44e14f0fb89c7f68a959b637" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43033 } } peers { permanent_uuid: "b140828ef67a4d0baeafb5df465a0d91" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 40937 } } peers { permanent_uuid: "ddd29755715042179bed0fb675ceeffe" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 36071 } }
I20260809 07:51:20.095091 2825 leader_election.cc:288] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033), b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937)
W20260809 07:51:20.095829 2363 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5100aadf44e14f0fb89c7f68a959b637 (127.18.71.67:43033): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
W20260809 07:51:20.095958 2362 leader_election.cc:341] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b140828ef67a4d0baeafb5df465a0d91 (127.18.71.66:40937): Not found: Tablet not found: 0a1d7340e84e4ec08978ce3cdea8b8aa
I20260809 07:51:20.096060 2362 leader_election.cc:302] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddd29755715042179bed0fb675ceeffe; no voters: 5100aadf44e14f0fb89c7f68a959b637, b140828ef67a4d0baeafb5df465a0d91
I20260809 07:51:20.096241 2825 raft_consensus.cc:2726] T 0a1d7340e84e4ec08978ce3cdea8b8aa P ddd29755715042179bed0fb675ceeffe [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:51:20.145260 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:20.144795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:48901)
0809 07:51:20.145084 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:20.145144 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":95,"threads_started":1}
I20260809 07:51:20.323328 2827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:20.322906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:44781)
0809 07:51:20.323166 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:20.323218 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
I20260809 07:51:20.514403 2828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:20.513988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:55401)
0809 07:51:20.514253 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:20.514310 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":92,"threads_started":1}
I20260809 07:51:21.146477 2829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:21.146029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:52739)
0809 07:51:21.146310 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:21.146368 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":114,"threads_started":1}
I20260809 07:51:21.324594 2832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:21.324094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:34909)
0809 07:51:21.324393 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:21.324498 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":116,"threads_started":1}
I20260809 07:51:21.515537 2834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:21.515142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:50279)
0809 07:51:21.515385 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:21.515441 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260809 07:51:22.147614 2835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:22.147214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:47913)
0809 07:51:22.147462 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:22.147517 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260809 07:51:22.325805 2836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:22.325335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:49131)
0809 07:51:22.325604 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:22.325683 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":82,"threads_started":1}
I20260809 07:51:22.516798 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:22.516302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.67:57009)
0809 07:51:22.516646 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:22.516702 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":88,"threads_started":1}
I20260809 07:51:23.149062 2838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:23.148633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.65:37321)
0809 07:51:23.148884 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:23.148945 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260809 07:51:23.327098 2839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:23.326697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33631 (local address 127.18.71.66:33553)
0809 07:51:23.326934 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:23.326987 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":76,"threads_started":1}
I20260809 07:51:23.499215 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 2305
I20260809 07:51:23.509123 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 2486
I20260809 07:51:23.516726 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 2633
2026-08-09T07:51:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19607 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:51:23 dist-test-slave-10pc krb5kdc[2842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:51:23 dist-test-slave-10pc krb5kdc[2842](info): set up 2 sockets
Aug 09 07:51:23 dist-test-slave-10pc krb5kdc[2842](info): commencing operation
krb5kdc: starting...
W20260809 07:51:24.601547 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:51:24 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:51:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:51:24.736855 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33033
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:24.849974 2856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:24.850234 2856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:24.850273 2856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:24.854139 2856 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 07:51:24.854183 2856 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 07:51:24.854198 2856 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 07:51:24.854212 2856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:24.854229 2856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:24.854247 2856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:24.854262 2856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:24.854285 2856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:24.859364 2856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:24.860360 2856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:24.861423 2856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:51:24.870477 2856 server_base.cc:1034] running on GCE node
W20260809 07:51:24.870493 2862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:24.870497 2861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:24.870656 2864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:24.871364 2856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:24.872491 2856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:24.873673 2856 hybrid_clock.cc:648] HybridClock initialized: now 1786261884873635 us; error 49 us; skew 500 ppm
Aug 09 07:51:24 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:24.876616 2856 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:51:24.877547 2856 webserver.cc:460] Webserver started at http://127.18.71.126:41697/ using document root <none> and password file <none>
I20260809 07:51:24.878084 2856 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:24.878144 2856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:24.878319 2856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:24.880337 2856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "9fc6cb9315d749fea595e60f6a1e39eb"
format_stamp: "Formatted at 2026-08-09 07:51:24 on dist-test-slave-10pc"
server_key: "35446d2a95cbdd20b15df7c687d49b6d"
server_key_iv: "2bbf1001ed7a8fcb95ca6e34cf04a8e4"
server_key_version: "encryptionkey@0"
I20260809 07:51:24.880939 2856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "9fc6cb9315d749fea595e60f6a1e39eb"
format_stamp: "Formatted at 2026-08-09 07:51:24 on dist-test-slave-10pc"
server_key: "35446d2a95cbdd20b15df7c687d49b6d"
server_key_iv: "2bbf1001ed7a8fcb95ca6e34cf04a8e4"
server_key_version: "encryptionkey@0"
I20260809 07:51:24.884682 2856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:51:24.887089 2871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:24.888525 2856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:24.888681 2856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "9fc6cb9315d749fea595e60f6a1e39eb"
format_stamp: "Formatted at 2026-08-09 07:51:24 on dist-test-slave-10pc"
server_key: "35446d2a95cbdd20b15df7c687d49b6d"
server_key_iv: "2bbf1001ed7a8fcb95ca6e34cf04a8e4"
server_key_version: "encryptionkey@0"
I20260809 07:51:24.888788 2856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:24.902228 2856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:24.905750 2856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:24.905952 2856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:24.913260 2856 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33033
I20260809 07:51:24.913595 2927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33033 every 8 connection(s)
I20260809 07:51:24.914366 2856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:24.917177 2928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:24.922636 2928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Bootstrap starting.
I20260809 07:51:24.924175 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 2856
I20260809 07:51:24.924317 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:51:24.924657 18717 external_mini_cluster.cc:1428] Setting key 1f6e4700bfe1f70a9b77ddecadfeb147
I20260809 07:51:24.925375 2928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:24.926286 2928 log.cc:824] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:24.928305 2928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: No bootstrap required, opened a new log
I20260809 07:51:24.931938 2928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:24.932220 2928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:24.932289 2928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc6cb9315d749fea595e60f6a1e39eb, State: Initialized, Role: FOLLOWER
I20260809 07:51:24.932770 2928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:24.932934 2928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:51:24.933025 2928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:51:24.933127 2928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 0 FOLLOWER]: Advancing to term 1
Aug 09 07:51:24 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:24.934116 2928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:24.934466 2928 leader_election.cc:302] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fc6cb9315d749fea595e60f6a1e39eb; no voters:
I20260809 07:51:24.934825 2928 leader_election.cc:288] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:51:24.934937 2933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:24.935158 2933 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 LEADER]: Becoming Leader. State: Replica: 9fc6cb9315d749fea595e60f6a1e39eb, State: Running, Role: LEADER
I20260809 07:51:24.935585 2933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:24.936221 2928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:24.936913 2933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fc6cb9315d749fea595e60f6a1e39eb. Latest consensus state: current_term: 1 leader_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } } }
I20260809 07:51:24.937130 2933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:24.937402 2934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } } }
I20260809 07:51:24.937511 2934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:24.937736 2943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:51:24.940550 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:24.925780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.18.71.126:33033)
0809 07:51:24.926247 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:24.926263 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:24.926298 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:51:24.927195 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:24.927217 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:24.927545 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:24.927890 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:24.928849 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:24.929835 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:24.930700 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:24.931180 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:24.931354 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:24.934275 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:24.934282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:24.934301 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:24.934339 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:24.936429 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:24.937241 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:24.937247 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:24.937252 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:24.937410 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:24.937755 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:24.937759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:24.937761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:24.938277 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:24.938484 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:24.939258 (+ 774us) server_negotiation.cc:299] Negotiation successful
0809 07:51:24.939560 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":147,"threads_started":1}
I20260809 07:51:24.941111 2943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:51:24.946900 2943 catalog_manager.cc:1347] Generated new cluster ID: 335f9420fc7a4bc196888e0d3f8091c8
I20260809 07:51:24.946986 2943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:51:24.956197 2943 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:51:24.957216 2943 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:51:24.961783 2943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Generated new TSK 0
I20260809 07:51:24.962456 2943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:51:24.997814 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33033
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:25.119632 2955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:25.119923 2955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:25.119968 2955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:25.124015 2955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:25.124082 2955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:25.124140 2955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:51:25.129397 2955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:25.130424 2955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:25.131408 2955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:25.141613 2965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:25.141625 2962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:25.141669 2955 server_base.cc:1034] running on GCE node
W20260809 07:51:25.141628 2961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:25.142299 2955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:25.142900 2955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:25.144061 2955 hybrid_clock.cc:648] HybridClock initialized: now 1786261885144025 us; error 48 us; skew 500 ppm
Aug 09 07:51:25 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:25.146981 2955 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:51:25.147825 2955 webserver.cc:460] Webserver started at http://127.18.71.65:42941/ using document root <none> and password file <none>
I20260809 07:51:25.148358 2955 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:25.148416 2955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:25.148653 2955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:25.150704 2955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "02613b4ab26b41199ee3a800c533a8a1"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "c5b6a8ec353d6e6336754d64955f6ab2"
server_key_iv: "81af6814ad61a570d88016466d9f75b5"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.151242 2955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "02613b4ab26b41199ee3a800c533a8a1"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "c5b6a8ec353d6e6336754d64955f6ab2"
server_key_iv: "81af6814ad61a570d88016466d9f75b5"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.154934 2955 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:51:25.157449 2972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.158735 2955 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:51:25.158881 2955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "02613b4ab26b41199ee3a800c533a8a1"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "c5b6a8ec353d6e6336754d64955f6ab2"
server_key_iv: "81af6814ad61a570d88016466d9f75b5"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.158999 2955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:25.172052 2955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:25.175009 2955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:25.175283 2955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:25.175925 2955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:25.176906 2955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:25.176996 2955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.177068 2955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:25.177119 2955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.186352 2955 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:40467
I20260809 07:51:25.186373 3094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:40467 every 8 connection(s)
I20260809 07:51:25.187307 2955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:25.195858 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 2955
I20260809 07:51:25.196004 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:51:25.196333 18717 external_mini_cluster.cc:1428] Setting key ef9c82c61f1744491c5f674ebf754098
Aug 09 07:51:25 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:25.203761 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.189269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:36341 (local address 127.18.71.126:33033)
0809 07:51:25.189445 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:25.189450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:25.190290 (+ 840us) server_negotiation.cc:407] Connection header received
0809 07:51:25.191726 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:25.191728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:25.191776 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:25.191853 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:25.193193 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.193826 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.194929 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.195147 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.195224 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.199869 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:25.199872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:25.199874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:25.199898 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:25.201486 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.202072 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.202076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.202078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.202168 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.202537 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.202542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.202544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.202768 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:25.202879 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:25.203504 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 07:51:25.203636 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 07:51:25.205186 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.189678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:36341)
0809 07:51:25.190122 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:25.190169 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:25.191495 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:25.192024 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:25.192034 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:25.192525 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:25.193051 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.193066 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.193940 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.193942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.194826 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.194835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.195277 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.195280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.195418 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.196115 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:25.196143 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:25.197696 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:25.201630 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:25.201637 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:25.201654 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:25.201960 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:25.202271 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:25.202274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:25.202276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:25.202428 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:25.202879 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:25.202887 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:25.203210 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 07:51:25.203449 (+ 239us) client_negotiation.cc:240] Negotiation successful
0809 07:51:25.203872 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":113,"threads_started":1}
I20260809 07:51:25.206580 3095 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33033
I20260809 07:51:25.206851 3095 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:51:25.207392 3095 heartbeater.cc:511] Master 127.18.71.126:33033 requested a full tablet report, sending...
I20260809 07:51:25.209105 2888 ts_manager.cc:194] Registered new tserver with Master: 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
I20260809 07:51:25.210425 2888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:36341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:51:25.227365 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33033
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:25.341419 3101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:25.341655 3101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:25.341704 3101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:25.345731 3101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:25.345786 3101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:25.345841 3101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:51:25.350936 3101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:25.351939 3101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:25.352969 3101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:25.363095 3109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:25.363225 3107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:25.363322 3106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:25.363528 3101 server_base.cc:1034] running on GCE node
I20260809 07:51:25.364145 3101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:25.364787 3101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:25.365972 3101 hybrid_clock.cc:648] HybridClock initialized: now 1786261885365944 us; error 50 us; skew 500 ppm
Aug 09 07:51:25 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:25.368738 3101 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:51:25.369676 3101 webserver.cc:460] Webserver started at http://127.18.71.66:41169/ using document root <none> and password file <none>
I20260809 07:51:25.370255 3101 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:25.370349 3101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:25.370589 3101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:25.372725 3101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "b8040b2316874780bd02b96d4413acf3"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "9376924841ae7472fb61cf7d89dd1a5a"
server_key_iv: "141c395cb3241e3b24247cb7da96d887"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.373332 3101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "b8040b2316874780bd02b96d4413acf3"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "9376924841ae7472fb61cf7d89dd1a5a"
server_key_iv: "141c395cb3241e3b24247cb7da96d887"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.377023 3101 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:25.379495 3116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.380836 3101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:51:25.380977 3101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b8040b2316874780bd02b96d4413acf3"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "9376924841ae7472fb61cf7d89dd1a5a"
server_key_iv: "141c395cb3241e3b24247cb7da96d887"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.381090 3101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:25.399936 3101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:25.403402 3101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:25.403617 3101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:25.404264 3101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:25.405200 3101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:25.405249 3101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.405290 3101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:25.405305 3101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.414562 3101 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:35299
I20260809 07:51:25.414577 3242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:35299 every 8 connection(s)
I20260809 07:51:25.415578 3101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:25.424799 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 3101
I20260809 07:51:25.424965 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
Aug 09 07:51:25 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:25.425284 18717 external_mini_cluster.cc:1428] Setting key b95cb8626b845e58d14be557a3f73070
I20260809 07:51:25.429527 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.417482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:49493 (local address 127.18.71.126:33033)
0809 07:51:25.417623 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:25.417627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:25.418391 (+ 764us) server_negotiation.cc:407] Connection header received
0809 07:51:25.419777 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:25.419781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:25.419844 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:25.419929 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:25.421153 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.421706 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.422871 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.423111 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.423225 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.425892 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:25.425896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:25.425899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:25.425927 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:25.427439 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.427953 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.427956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.427958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.428049 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.428402 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.428406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.428407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.428621 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:25.428737 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:25.429239 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 07:51:25.429361 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:51:25.430392 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.417782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:49493)
0809 07:51:25.418233 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:25.418276 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:25.419553 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:25.420128 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:25.420140 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:25.420517 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:25.420994 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.421007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.421843 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.421847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.422737 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.422746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.423266 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.423270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.423444 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.424019 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:25.424035 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:25.425671 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:25.427562 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:25.427568 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:25.427582 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:25.427842 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:25.428167 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:25.428169 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:25.428171 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:25.428302 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:25.428752 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:25.428760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:25.428988 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 07:51:25.429181 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 07:51:25.429495 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":124,"threads_started":1}
I20260809 07:51:25.431567 3243 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33033
I20260809 07:51:25.431816 3243 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:25.432358 3243 heartbeater.cc:511] Master 127.18.71.126:33033 requested a full tablet report, sending...
I20260809 07:51:25.433490 2888 ts_manager.cc:194] Registered new tserver with Master: b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299)
I20260809 07:51:25.434058 2888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:49493
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:51:25.456904 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33033
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:25.574906 3248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:25.575170 3248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:25.575213 3248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:25.579226 3248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:25.579298 3248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:25.579356 3248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:51:25.584856 3248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:25.586050 3248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:25.587193 3248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:25.597265 3255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:25.597472 3248 server_base.cc:1034] running on GCE node
W20260809 07:51:25.597258 3254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:25.597258 3257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:25.598045 3248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:25.598744 3248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:25.599965 3248 hybrid_clock.cc:648] HybridClock initialized: now 1786261885599942 us; error 29 us; skew 500 ppm
Aug 09 07:51:25 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:25.603011 3248 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:51:25.603965 3248 webserver.cc:460] Webserver started at http://127.18.71.67:46205/ using document root <none> and password file <none>
I20260809 07:51:25.604658 3248 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:25.604759 3248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:25.605047 3248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:51:25.607204 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "2183d147333749f5b4bf990fb5904774"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "369657a4ec9babc9c3bb7f0bba5c9f44"
server_key_iv: "48e7bbf9bdc2e3c792abcb5652962a74"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.607820 3248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "2183d147333749f5b4bf990fb5904774"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "369657a4ec9babc9c3bb7f0bba5c9f44"
server_key_iv: "48e7bbf9bdc2e3c792abcb5652962a74"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.611707 3248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 07:51:25.614341 3265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.615659 3248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:25.615789 3248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "2183d147333749f5b4bf990fb5904774"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "369657a4ec9babc9c3bb7f0bba5c9f44"
server_key_iv: "48e7bbf9bdc2e3c792abcb5652962a74"
server_key_version: "encryptionkey@0"
I20260809 07:51:25.615913 3248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:25.646517 3248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:25.650159 3248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:25.650395 3248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:25.651046 3248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:25.652359 3248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:25.652428 3248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.652511 3248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:25.652544 3248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:25.662083 3248 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:33199
I20260809 07:51:25.662106 3390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:33199 every 8 connection(s)
I20260809 07:51:25.663193 3248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:25.665124 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 3248
I20260809 07:51:25.665261 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:51:25.665557 18717 external_mini_cluster.cc:1428] Setting key 1cbc7d8ec6b181e3e99155219076b56e
Aug 09 07:51:25 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261885, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:25.677692 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.665188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:41895 (local address 127.18.71.126:33033)
0809 07:51:25.665352 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:25.665357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:25.666271 (+ 914us) server_negotiation.cc:407] Connection header received
0809 07:51:25.667940 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:25.667943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:25.668001 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:25.668102 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:25.669476 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.670022 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.671194 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.671425 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.671535 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.674106 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:25.674110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:25.674112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:25.674139 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:25.675527 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.676063 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.676066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.676069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.676157 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.676540 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.676544 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.676546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.676741 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:25.676855 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:25.677377 (+ 522us) server_negotiation.cc:299] Negotiation successful
0809 07:51:25.677503 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 07:51:25.678700 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.665608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:41895)
0809 07:51:25.666108 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:25.666174 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:25.667705 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:25.668309 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:25.668317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:25.668795 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:25.669329 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.669344 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.670160 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.670164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.671055 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.671064 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.671580 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.671584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.671765 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.672358 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:25.672375 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:25.673929 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:25.675653 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:25.675662 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:25.675678 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:25.675952 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:25.676254 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:25.676257 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:25.676259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:25.676407 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:25.676856 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:25.676864 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:25.677117 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 07:51:25.677315 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 07:51:25.677654 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":148,"threads_started":1}
I20260809 07:51:25.679751 3391 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33033
I20260809 07:51:25.680009 3391 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:25.680589 3391 heartbeater.cc:511] Master 127.18.71.126:33033 requested a full tablet report, sending...
I20260809 07:51:25.681617 2888 ts_manager.cc:194] Registered new tserver with Master: 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199)
I20260809 07:51:25.682190 2888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:41895
I20260809 07:51:25.691814 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:51:25.701918 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.694233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.18.71.126:33033)
0809 07:51:25.694357 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:25.694367 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:25.694574 (+ 207us) server_negotiation.cc:407] Connection header received
0809 07:51:25.694668 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:25.694670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:25.694709 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:25.694777 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:25.695455 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.695954 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.696858 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.697068 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.697158 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.698608 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:25.698610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:25.698612 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:25.698630 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:25.700397 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.700846 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.700849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.700850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.700926 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:25.701179 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:25.701181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:25.701182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:25.701395 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:25.701499 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:25.701703 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 07:51:25.701798 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:51:25.704769 2888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:51:25.707881 2888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:51:25.723219 3404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.717369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35299 (local address 127.0.0.1:54684)
0809 07:51:25.717789 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:25.717804 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:25.717916 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:25.718864 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:25.718867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:25.718882 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:25.718992 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.719004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.720674 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.720677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.721548 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.721557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.722596 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.722603 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.722776 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.722831 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 07:51:25.722950 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 07:51:25.723083 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":42,"thread_start_us":141,"threads_started":1}
I20260809 07:51:25.723229 3405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.718095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.18.71.66:35299)
0809 07:51:25.718413 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:25.718422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:25.718441 (+ 19us) server_negotiation.cc:407] Connection header received
0809 07:51:25.718504 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:25.718507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:25.718711 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:25.718952 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:25.719124 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.720407 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.721688 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.722459 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.722737 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.722883 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:25.722972 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 07:51:25.723044 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":117,"threads_started":1}
I20260809 07:51:25.724505 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.717282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40467 (local address 127.0.0.1:54172)
0809 07:51:25.717723 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:25.717783 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:25.717913 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:25.718712 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:25.718718 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:25.718743 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:25.719098 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.719105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.720579 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.720584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.723007 (+ 2423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.723016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.724056 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.724062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.724161 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.724185 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:51:25.724266 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 07:51:25.724335 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":1118,"thread_start_us":81,"threads_started":1}
I20260809 07:51:25.724577 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.717418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54172 (local address 127.18.71.65:40467)
0809 07:51:25.718203 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:25.718213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:25.718240 (+ 27us) server_negotiation.cc:407] Connection header received
0809 07:51:25.718321 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:25.718326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:25.718536 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:25.718698 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:25.719228 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.720407 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.723129 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.723938 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.724055 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.724202 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:25.724311 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 07:51:25.724403 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":94,"threads_started":1}
I20260809 07:51:25.725567 3406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.718881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33199 (local address 127.0.0.1:40620)
0809 07:51:25.719809 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:25.719823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:25.719912 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:25.720436 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:25.720439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:25.720497 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:25.720616 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.720624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.721599 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.721605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.723838 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:25.723848 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.725198 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.725202 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:25.725284 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.725304 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 07:51:25.725368 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 07:51:25.725433 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":122,"thread_start_us":97,"threads_started":1}
I20260809 07:51:25.725785 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:25.719139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.18.71.67:33199)
0809 07:51:25.719517 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:25.719529 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:25.720022 (+ 493us) server_negotiation.cc:407] Connection header received
0809 07:51:25.720130 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:25.720135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:25.720308 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:25.720458 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:25.720737 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.721469 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.723956 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:25.725065 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:25.725198 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:25.725339 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:25.725394 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 07:51:25.725477 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":177,"threads_started":1}
I20260809 07:51:25.726763 3168 tablet_service.cc:1467] Processing CreateTablet for tablet fac3c4ba6997465986b5ef6f1534bb84 (DEFAULT_TABLE table=test-table [id=2228586e61f74812a88239f67b270d9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:25.727516 3022 tablet_service.cc:1467] Processing CreateTablet for tablet fac3c4ba6997465986b5ef6f1534bb84 (DEFAULT_TABLE table=test-table [id=2228586e61f74812a88239f67b270d9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:25.727996 3168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac3c4ba6997465986b5ef6f1534bb84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:25.728005 3315 tablet_service.cc:1467] Processing CreateTablet for tablet fac3c4ba6997465986b5ef6f1534bb84 (DEFAULT_TABLE table=test-table [id=2228586e61f74812a88239f67b270d9a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:25.728739 3022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac3c4ba6997465986b5ef6f1534bb84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:25.728987 3315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac3c4ba6997465986b5ef6f1534bb84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:25.734869 3408 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Bootstrap starting.
I20260809 07:51:25.736318 3409 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Bootstrap starting.
I20260809 07:51:25.736768 3410 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap starting.
I20260809 07:51:25.737372 3408 tablet_bootstrap.cc:654] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:25.738792 3410 tablet_bootstrap.cc:654] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:25.739054 3408 log.cc:824] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:25.739269 3409 tablet_bootstrap.cc:654] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:25.739698 3410 log.cc:824] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:25.740114 3409 log.cc:824] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:25.741338 3408 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: No bootstrap required, opened a new log
I20260809 07:51:25.741546 3408 ts_tablet_manager.cc:1397] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:51:25.741740 3410 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: No bootstrap required, opened a new log
I20260809 07:51:25.741940 3410 ts_tablet_manager.cc:1397] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:51:25.742113 3409 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: No bootstrap required, opened a new log
I20260809 07:51:25.742287 3409 ts_tablet_manager.cc:1397] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260809 07:51:25.745445 3410 raft_consensus.cc:348] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:25.745707 3410 raft_consensus.cc:374] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:25.745764 3410 raft_consensus.cc:729] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02613b4ab26b41199ee3a800c533a8a1, State: Initialized, Role: FOLLOWER
I20260809 07:51:25.745618 3408 raft_consensus.cc:348] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:25.745901 3408 raft_consensus.cc:374] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:25.745959 3408 raft_consensus.cc:729] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2183d147333749f5b4bf990fb5904774, State: Initialized, Role: FOLLOWER
I20260809 07:51:25.746281 3410 consensus_queue.cc:260] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:25.746379 3408 consensus_queue.cc:260] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:25.746974 3095 heartbeater.cc:503] Master 127.18.71.126:33033 was elected leader, sending a full tablet report...
I20260809 07:51:25.747080 3409 raft_consensus.cc:348] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:25.747339 3410 ts_tablet_manager.cc:1428] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 07:51:25.747359 3408 ts_tablet_manager.cc:1428] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:51:25.747346 3409 raft_consensus.cc:374] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:25.747412 3409 raft_consensus.cc:729] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8040b2316874780bd02b96d4413acf3, State: Initialized, Role: FOLLOWER
I20260809 07:51:25.747828 3391 heartbeater.cc:503] Master 127.18.71.126:33033 was elected leader, sending a full tablet report...
I20260809 07:51:25.747815 3409 consensus_queue.cc:260] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:25.748751 3409 ts_tablet_manager.cc:1428] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260809 07:51:25.749944 3243 heartbeater.cc:503] Master 127.18.71.126:33033 was elected leader, sending a full tablet report...
W20260809 07:51:25.914223 3392 tablet.cc:2369] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:51:25.916756 3244 tablet.cc:2369] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:51:25.938680 3096 tablet.cc:2369] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:26.037977 3416 raft_consensus.cc:482] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:26.038268 3416 raft_consensus.cc:504] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:26.039541 3416 leader_election.cc:288] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467), 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199)
I20260809 07:51:26.043646 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:26.039995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:48097 (local address 127.18.71.65:40467)
0809 07:51:26.040148 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:26.040153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:26.040300 (+ 147us) server_negotiation.cc:407] Connection header received
0809 07:51:26.040526 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:26.040529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:26.040587 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:26.040684 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:26.041036 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.041549 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.042635 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.043051 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.043193 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:26.043248 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:26.043446 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 07:51:26.043519 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:51:26.043679 3417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:26.039836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40467 (local address 127.18.71.66:48097)
0809 07:51:26.040197 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:26.040219 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:26.040380 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:26.040699 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:26.040703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:26.040731 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:26.040912 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:26.040918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.041679 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.041683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:26.042498 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:26.042507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.043191 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.043195 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:26.043296 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:26.043320 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:51:26.043425 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 07:51:26.043501 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":40,"spinlock_wait_cycles":42752,"thread_start_us":81,"threads_started":1}
I20260809 07:51:26.043836 3407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:26.039959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:51039 (local address 127.18.71.67:33199)
0809 07:51:26.040107 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:26.040112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:26.040312 (+ 200us) server_negotiation.cc:407] Connection header received
0809 07:51:26.040537 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:26.040540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:26.040608 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:26.040684 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:26.041052 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.041779 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.042850 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.043252 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.043391 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:26.043444 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:26.043653 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 07:51:26.043723 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:51:26.043862 3418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:26.039868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33199 (local address 127.18.71.66:51039)
0809 07:51:26.040203 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:26.040216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:26.040382 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:26.040703 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:26.040706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:26.040733 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:26.040925 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:26.040931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.041923 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.041929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:26.042715 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:26.042724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.043391 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.043395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:26.043516 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:26.043533 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 07:51:26.043633 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 07:51:26.043707 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":39,"spinlock_wait_cycles":30720,"thread_start_us":96,"threads_started":1}
I20260809 07:51:26.044380 3044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac3c4ba6997465986b5ef6f1534bb84" candidate_uuid: "b8040b2316874780bd02b96d4413acf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02613b4ab26b41199ee3a800c533a8a1" is_pre_election: true
I20260809 07:51:26.044509 3340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac3c4ba6997465986b5ef6f1534bb84" candidate_uuid: "b8040b2316874780bd02b96d4413acf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2183d147333749f5b4bf990fb5904774" is_pre_election: true
I20260809 07:51:26.044777 3340 raft_consensus.cc:2454] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8040b2316874780bd02b96d4413acf3 in term 0.
I20260809 07:51:26.044777 3044 raft_consensus.cc:2454] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8040b2316874780bd02b96d4413acf3 in term 0.
I20260809 07:51:26.045390 3117 leader_election.cc:302] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 02613b4ab26b41199ee3a800c533a8a1, 2183d147333749f5b4bf990fb5904774, b8040b2316874780bd02b96d4413acf3; no voters:
I20260809 07:51:26.045703 3416 raft_consensus.cc:2781] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:26.045790 3416 raft_consensus.cc:482] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:26.045822 3416 raft_consensus.cc:3037] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:26.046942 3416 raft_consensus.cc:504] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:26.047348 3416 leader_election.cc:288] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [CANDIDATE]: Term 1 election: Requested vote from peers 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467), 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199)
I20260809 07:51:26.047806 3340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac3c4ba6997465986b5ef6f1534bb84" candidate_uuid: "b8040b2316874780bd02b96d4413acf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2183d147333749f5b4bf990fb5904774"
I20260809 07:51:26.047837 3044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac3c4ba6997465986b5ef6f1534bb84" candidate_uuid: "b8040b2316874780bd02b96d4413acf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02613b4ab26b41199ee3a800c533a8a1"
I20260809 07:51:26.047947 3340 raft_consensus.cc:3037] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:26.047947 3044 raft_consensus.cc:3037] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:26.049337 3044 raft_consensus.cc:2454] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8040b2316874780bd02b96d4413acf3 in term 1.
I20260809 07:51:26.049337 3340 raft_consensus.cc:2454] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8040b2316874780bd02b96d4413acf3 in term 1.
I20260809 07:51:26.049695 3119 leader_election.cc:302] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2183d147333749f5b4bf990fb5904774, b8040b2316874780bd02b96d4413acf3; no voters:
I20260809 07:51:26.049896 3416 raft_consensus.cc:2781] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:26.050149 3416 raft_consensus.cc:686] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 1 LEADER]: Becoming Leader. State: Replica: b8040b2316874780bd02b96d4413acf3, State: Running, Role: LEADER
I20260809 07:51:26.050527 3416 consensus_queue.cc:237] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:26.053867 2886 catalog_manager.cc:5495] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8040b2316874780bd02b96d4413acf3 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "b8040b2316874780bd02b96d4413acf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } health_report { overall_health: UNKNOWN } } }
I20260809 07:51:26.090770 3405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:26.087349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54690 (local address 127.18.71.66:35299)
0809 07:51:26.087488 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:26.087492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:26.087506 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:51:26.087654 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:26.087657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:26.087720 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:26.087813 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:51:26.088060 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.088630 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.089454 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:26.089666 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:26.089740 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:26.090014 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:51:26.090454 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:51:26.090547 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:26.090605 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 07:51:26.090658 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:51:26.097242 3044 raft_consensus.cc:1261] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Refusing update from remote peer b8040b2316874780bd02b96d4413acf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:26.097242 3340 raft_consensus.cc:1261] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 1 FOLLOWER]: Refusing update from remote peer b8040b2316874780bd02b96d4413acf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:26.098094 3419 consensus_queue.cc:1035] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:26.098307 3416 consensus_queue.cc:1035] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:26.106783 3424 mvcc.cc:205] Tried to move back new op lower bound from 7316528685445779456 to 7316528685265805312. Current Snapshot: MvccSnapshot[applied={T|T < 7316528685445779456}]
I20260809 07:51:26.108253 3422 mvcc.cc:205] Tried to move back new op lower bound from 7316528685445779456 to 7316528685265805312. Current Snapshot: MvccSnapshot[applied={T|T < 7316528685445779456}]
I20260809 07:51:26.108417 3423 mvcc.cc:205] Tried to move back new op lower bound from 7316528685445779456 to 7316528685265805312. Current Snapshot: MvccSnapshot[applied={T|T < 7316528685445779456}]
I20260809 07:51:26.115921 2886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:26.116067 2886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:26.118623 2886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Sending DeleteTablet for 3 replicas of tablet fac3c4ba6997465986b5ef6f1534bb84
I20260809 07:51:26.119459 3315 tablet_service.cc:1514] Processing DeleteTablet for tablet fac3c4ba6997465986b5ef6f1534bb84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:26 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40620
I20260809 07:51:26.119465 3022 tablet_service.cc:1514] Processing DeleteTablet for tablet fac3c4ba6997465986b5ef6f1534bb84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:26 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:54172
I20260809 07:51:26.119549 3168 tablet_service.cc:1514] Processing DeleteTablet for tablet fac3c4ba6997465986b5ef6f1534bb84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:26 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:54684
I20260809 07:51:26.119915 3434 tablet_replica.cc:330] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: stopping tablet replica
I20260809 07:51:26.119989 3435 tablet_replica.cc:330] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: stopping tablet replica
I20260809 07:51:26.120288 3435 raft_consensus.cc:2229] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:51:26.120303 3434 raft_consensus.cc:2229] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:26.120623 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 2955
I20260809 07:51:26.120801 3435 raft_consensus.cc:2258] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:26.125931 3435 ts_tablet_manager.cc:1905] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:26.128842 3435 ts_tablet_manager.cc:1918] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:26.128931 3435 log.cc:1196] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/fac3c4ba6997465986b5ef6f1534bb84
I20260809 07:51:26.129259 3435 ts_tablet_manager.cc:1939] T fac3c4ba6997465986b5ef6f1534bb84 P b8040b2316874780bd02b96d4413acf3: Deleting consensus metadata
I20260809 07:51:26.130556 2875 catalog_manager.cc:4841] TS b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299): tablet fac3c4ba6997465986b5ef6f1534bb84 (table test-table [id=2228586e61f74812a88239f67b270d9a]) successfully deleted
I20260809 07:51:26.120687 3432 tablet_replica.cc:330] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: stopping tablet replica
I20260809 07:51:26.133229 3432 raft_consensus.cc:2229] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:26.133567 3432 raft_consensus.cc:2258] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:26.134958 3432 ts_tablet_manager.cc:1905] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:26.137490 3432 ts_tablet_manager.cc:1918] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:26.137606 3432 log.cc:1196] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/fac3c4ba6997465986b5ef6f1534bb84
I20260809 07:51:26.137943 3432 ts_tablet_manager.cc:1939] T fac3c4ba6997465986b5ef6f1534bb84 P 2183d147333749f5b4bf990fb5904774: Deleting consensus metadata
I20260809 07:51:26.138788 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 3101
W20260809 07:51:26.138909 2872 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:40467 (error 108)
I20260809 07:51:26.141359 2874 catalog_manager.cc:4841] TS 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199): tablet fac3c4ba6997465986b5ef6f1534bb84 (table test-table [id=2228586e61f74812a88239f67b270d9a]) successfully deleted
I20260809 07:51:26.142058 3406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:26.141766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40467 (local address 127.0.0.1:54178)
0809 07:51:26.141904 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:26.141982 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 07:51:26.142282 2872 catalog_manager.cc:4584] TS 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fac3c4ba6997465986b5ef6f1534bb84: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40467: connect: Connection refused (error 111)
I20260809 07:51:26.147047 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 3248
I20260809 07:51:26.155295 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 2856
I20260809 07:51:46.166750 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33033
--webserver_interface=127.18.71.126
--webserver_port=41697
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:51:46.301082 3437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:46.301443 3437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:46.301523 3437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:46.305953 3437 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 07:51:46.306025 3437 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 07:51:46.306041 3437 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 07:51:46.306056 3437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:51:46.306099 3437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:46.306133 3437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:46.306162 3437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:51:46.306190 3437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:51:46.312063 3437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=41697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:46.313195 3437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:46.314231 3437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:46.328302 3443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:46.328296 3444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:46.328291 3446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:46.328706 3437 server_base.cc:1034] running on GCE node
I20260809 07:51:46.329334 3437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:46.330453 3437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:46.331650 3437 hybrid_clock.cc:648] HybridClock initialized: now 1786261906331631 us; error 30 us; skew 500 ppm
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:46.334551 3437 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:51:46.335381 3437 webserver.cc:460] Webserver started at http://127.18.71.126:41697/ using document root <none> and password file <none>
I20260809 07:51:46.335923 3437 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:46.335997 3437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:46.339918 3437 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:46.342121 3453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.343375 3437 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:51:46.343537 3437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "9fc6cb9315d749fea595e60f6a1e39eb"
format_stamp: "Formatted at 2026-08-09 07:51:24 on dist-test-slave-10pc"
server_key: "35446d2a95cbdd20b15df7c687d49b6d"
server_key_iv: "2bbf1001ed7a8fcb95ca6e34cf04a8e4"
server_key_version: "encryptionkey@0"
I20260809 07:51:46.344141 3437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:46.365563 3437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:46.368631 3437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:46.368841 3437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:46.375864 3437 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33033
I20260809 07:51:46.376202 3510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33033 every 8 connection(s)
I20260809 07:51:46.377048 3437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:51:46.381479 3511 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:51:46.383177 3511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Bootstrap starting.
I20260809 07:51:46.384742 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 3437
I20260809 07:51:46.385200 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:40467
--local_ip_for_outbound_sockets=127.18.71.65
--tserver_master_addrs=127.18.71.126:33033
--webserver_port=42941
--webserver_interface=127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:46.400552 3511 log.cc:824] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:46.409410 3511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:46.409802 3511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Bootstrap complete.
I20260809 07:51:46.413977 3511 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:46.414361 3511 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc6cb9315d749fea595e60f6a1e39eb, State: Initialized, Role: FOLLOWER
I20260809 07:51:46.414891 3511 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:46.415041 3511 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:51:46.415126 3511 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:51:46.415251 3511 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 1 FOLLOWER]: Advancing to term 2
I20260809 07:51:46.417235 3511 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:46.417587 3511 leader_election.cc:302] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fc6cb9315d749fea595e60f6a1e39eb; no voters:
I20260809 07:51:46.417928 3511 leader_election.cc:288] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 07:51:46.418011 3515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 07:51:46.418217 3515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 LEADER]: Becoming Leader. State: Replica: 9fc6cb9315d749fea595e60f6a1e39eb, State: Running, Role: LEADER
I20260809 07:51:46.418512 3515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:51:46.419173 3511 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:51:46.419468 3515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fc6cb9315d749fea595e60f6a1e39eb. Latest consensus state: current_term: 2 leader_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } } }
I20260809 07:51:46.419595 3515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:46.419957 3521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:51:46.420153 3518 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } } }
I20260809 07:51:46.420339 3518 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: This master's current role is: LEADER
I20260809 07:51:46.425010 3521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:51:46.425637 3521 catalog_manager.cc:1259] Loaded cluster ID: 335f9420fc7a4bc196888e0d3f8091c8
I20260809 07:51:46.425701 3521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:51:46.428195 3521 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:51:46.428920 3521 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Loaded TSK: 0
I20260809 07:51:46.444489 3521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Generated new TSK 1
I20260809 07:51:46.444731 3521 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 07:51:46.503743 3513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:46.504031 3513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:46.504128 3513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:46.508139 3513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:46.508226 3513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:46.508313 3513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:51:46.513583 3513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:40467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=42941
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:46.514592 3513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:46.515596 3513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:46.525722 3544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:46.525734 3541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:46.525951 3513 server_base.cc:1034] running on GCE node
W20260809 07:51:46.525729 3542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:46.526456 3513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:46.527161 3513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:46.528335 3513 hybrid_clock.cc:648] HybridClock initialized: now 1786261906528306 us; error 44 us; skew 500 ppm
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:46.531219 3513 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:51:46.532250 3513 webserver.cc:460] Webserver started at http://127.18.71.65:42941/ using document root <none> and password file <none>
I20260809 07:51:46.532855 3513 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:46.532928 3513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:46.536806 3513 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:51:46.538952 3552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.540241 3513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:46.540330 3513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "02613b4ab26b41199ee3a800c533a8a1"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "c5b6a8ec353d6e6336754d64955f6ab2"
server_key_iv: "81af6814ad61a570d88016466d9f75b5"
server_key_version: "encryptionkey@0"
I20260809 07:51:46.540964 3513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:46.554322 3513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:46.557466 3513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:46.557659 3513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:46.558281 3513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:46.559455 3559 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 07:51:46.563659 3513 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 07:51:46.563719 3513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260809 07:51:46.563781 3513 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 07:51:46.565515 3513 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 07:51:46.565573 3513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.001s
I20260809 07:51:46.565848 3559 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap starting.
I20260809 07:51:46.576733 3513 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:40467
I20260809 07:51:46.576786 3679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:40467 every 8 connection(s)
I20260809 07:51:46.578118 3513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:51:46.582829 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 3513
I20260809 07:51:46.583386 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:35299
--local_ip_for_outbound_sockets=127.18.71.66
--tserver_master_addrs=127.18.71.126:33033
--webserver_port=41169
--webserver_interface=127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:51:46.588049 3559 log.cc:824] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:46.596329 3559 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:51:46.596817 3559 tablet_bootstrap.cc:492] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap complete.
I20260809 07:51:46.597847 3683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.580442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:59097)
0809 07:51:46.581011 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:46.581053 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:46.581914 (+ 861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:46.583287 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:46.583297 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:46.584059 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:46.584827 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.584850 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.586308 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.586311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.587216 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.587226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.588001 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.588005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.588220 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.588932 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:46.588958 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:46.590985 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:46.594013 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.594018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.594031 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.594385 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.594880 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.594883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.594885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.595067 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.595914 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:46.595920 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:46.596219 (+ 299us) client_negotiation.cc:769] Sending connection context
0809 07:51:46.596412 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 07:51:46.596802 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":152,"threads_started":1}
I20260809 07:51:46.598295 3559 ts_tablet_manager.cc:1397] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Time spent bootstrapping tablet: real 0.033s user 0.022s sys 0.008s
I20260809 07:51:46.598574 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.580399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:59097 (local address 127.18.71.126:33033)
0809 07:51:46.580826 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:46.580847 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:46.581174 (+ 327us) server_negotiation.cc:407] Connection header received
0809 07:51:46.582346 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:46.582358 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:46.582776 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:46.583201 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:46.585068 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.586132 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.587351 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.587848 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.588031 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.591264 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:46.591275 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:46.591297 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:46.591352 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:46.593883 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.594537 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.594547 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.594555 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.594765 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.595190 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.595195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.595200 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.595778 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:46.595937 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:46.596725 (+ 788us) server_negotiation.cc:299] Negotiation successful
0809 07:51:46.597157 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":178,"threads_started":1}
I20260809 07:51:46.601140 3680 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33033
I20260809 07:51:46.601392 3680 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:46.601042 3559 raft_consensus.cc:348] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:46.601608 3559 raft_consensus.cc:729] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02613b4ab26b41199ee3a800c533a8a1, State: Initialized, Role: FOLLOWER
I20260809 07:51:46.601935 3680 heartbeater.cc:511] Master 127.18.71.126:33033 requested a full tablet report, sending...
I20260809 07:51:46.602128 3559 consensus_queue.cc:260] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } }
I20260809 07:51:46.603134 3559 ts_tablet_manager.cc:1428] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 07:51:46.604673 3472 ts_manager.cc:194] Registered new tserver with Master: 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467)
I20260809 07:51:46.608397 3472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:59097
I20260809 07:51:46.611492 3680 heartbeater.cc:503] Master 127.18.71.126:33033 was elected leader, sending a full tablet report...
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.65@KRBTEST.COM
I20260809 07:51:46.617962 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.607488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45748 (local address 127.18.71.65:40467)
0809 07:51:46.608162 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:46.608173 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:46.608193 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:51:46.608255 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:46.608260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:46.608500 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:46.608652 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:46.609537 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.610333 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.611242 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.611637 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.611751 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.614527 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:46.614535 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:46.614559 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:46.614581 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:46.616143 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.616698 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.616702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.616703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.616780 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.617055 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.617058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.617059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.617266 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:46.617364 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:46.617724 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 07:51:46.617849 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":102,"threads_started":1}
I20260809 07:51:46.618007 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.607403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40467 (local address 127.0.0.1:45748)
0809 07:51:46.607839 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:46.607890 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:46.608016 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:46.608822 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:46.608828 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:46.609267 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:46.609399 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.609407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.610453 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.610458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.611114 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.611128 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.611765 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.611769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.612058 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.612670 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:46.612682 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:46.614399 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:46.616251 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.616261 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.616267 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.616597 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.616868 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.616871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.616873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.616968 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.617374 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:46.617380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:46.617501 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 07:51:46.617655 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 07:51:46.617846 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":140,"threads_started":1}
I20260809 07:51:46.619130 3604 tablet_service.cc:1514] Processing DeleteTablet for tablet fac3c4ba6997465986b5ef6f1534bb84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:26 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:45748
I20260809 07:51:46.620103 3689 tablet_replica.cc:330] stopping tablet replica
I20260809 07:51:46.620401 3689 raft_consensus.cc:2229] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:46.620694 3689 raft_consensus.cc:2258] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:46.621722 3689 ts_tablet_manager.cc:1905] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:46.627686 3689 ts_tablet_manager.cc:1918] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:46.627775 3689 log.cc:1196] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal/wals/fac3c4ba6997465986b5ef6f1534bb84
I20260809 07:51:46.628062 3689 ts_tablet_manager.cc:1939] T fac3c4ba6997465986b5ef6f1534bb84 P 02613b4ab26b41199ee3a800c533a8a1: Deleting consensus metadata
I20260809 07:51:46.629771 3455 catalog_manager.cc:4841] TS 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467): tablet fac3c4ba6997465986b5ef6f1534bb84 (table test-table [id=2228586e61f74812a88239f67b270d9a]) successfully deleted
W20260809 07:51:46.706960 3684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:46.707257 3684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:46.707329 3684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:46.711303 3684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:46.711388 3684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:46.711483 3684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:51:46.716615 3684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:35299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=41169
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:46.717593 3684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:46.718564 3684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:46.728152 3695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:46.728117 3694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:46.728117 3698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:46.728240 3684 server_base.cc:1034] running on GCE node
I20260809 07:51:46.728783 3684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:46.729460 3684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:46.730618 3684 hybrid_clock.cc:648] HybridClock initialized: now 1786261906730603 us; error 24 us; skew 500 ppm
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:46.733544 3684 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:51:46.734413 3684 webserver.cc:460] Webserver started at http://127.18.71.66:41169/ using document root <none> and password file <none>
I20260809 07:51:46.734962 3684 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:46.735083 3684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:46.738842 3684 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 07:51:46.740859 3705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.742055 3684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:46.742195 3684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b8040b2316874780bd02b96d4413acf3"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "9376924841ae7472fb61cf7d89dd1a5a"
server_key_iv: "141c395cb3241e3b24247cb7da96d887"
server_key_version: "encryptionkey@0"
I20260809 07:51:46.742728 3684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:46.760004 3684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:46.763356 3684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:46.763561 3684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:46.764151 3684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:46.765005 3684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:46.765079 3684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.765156 3684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:46.765193 3684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.774358 3684 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:35299
I20260809 07:51:46.774391 3828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:35299 every 8 connection(s)
I20260809 07:51:46.775338 3684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:51:46.783753 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 3684
I20260809 07:51:46.784191 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:33199
--local_ip_for_outbound_sockets=127.18.71.67
--tserver_master_addrs=127.18.71.126:33033
--webserver_port=46205
--webserver_interface=127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:46.789984 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.777125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:45775 (local address 127.18.71.126:33033)
0809 07:51:46.777285 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:46.777290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:46.778177 (+ 887us) server_negotiation.cc:407] Connection header received
0809 07:51:46.779518 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:46.779521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:46.779589 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:46.779681 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:46.780928 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.781448 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.782537 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.782761 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.782835 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.785704 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:46.785707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:46.785709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:46.785732 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:46.787237 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.787976 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.787980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.787981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.788106 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.788519 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.788524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.788528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.788787 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:46.788868 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:46.789577 (+ 709us) server_negotiation.cc:299] Negotiation successful
0809 07:51:46.789733 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 07:51:46.790783 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.777553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:45775)
0809 07:51:46.778031 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:46.778074 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:46.779324 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:46.779869 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:46.779876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:46.780226 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:46.780797 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.780811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.781549 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.781551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.782435 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.782444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.782891 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.782894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.783036 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.783627 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:46.783644 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:46.785526 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:46.787375 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.787383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.787401 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.787799 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.788196 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.788198 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.788224 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.788355 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.788962 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:46.788969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:46.789291 (+ 322us) client_negotiation.cc:769] Sending connection context
0809 07:51:46.789487 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 07:51:46.789830 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":122,"threads_started":1}
I20260809 07:51:46.791947 3829 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33033
I20260809 07:51:46.792229 3829 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:46.792889 3829 heartbeater.cc:511] Master 127.18.71.126:33033 requested a full tablet report, sending...
I20260809 07:51:46.794135 3472 ts_manager.cc:194] Registered new tserver with Master: b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299)
I20260809 07:51:46.794723 3472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:45775
W20260809 07:51:46.902771 3832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:51:46.903034 3832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:51:46.903095 3832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:51:46.907253 3832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:51:46.907325 3832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:51:46.907416 3832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:51:46.912853 3832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:33199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=46205
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:51:46.913903 3832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:51:46.914947 3832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:51:46.924865 3842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:51:46.924865 3839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:46.925071 3832 server_base.cc:1034] running on GCE node
W20260809 07:51:46.924929 3840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:51:46.925532 3832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:51:46.926151 3832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:51:46.927313 3832 hybrid_clock.cc:648] HybridClock initialized: now 1786261906927281 us; error 44 us; skew 500 ppm
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:51:46.930109 3832 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:51:46.930967 3832 webserver.cc:460] Webserver started at http://127.18.71.67:46205/ using document root <none> and password file <none>
I20260809 07:51:46.931540 3832 fs_manager.cc:359] Metadata directory not provided
I20260809 07:51:46.931643 3832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:51:46.935472 3832 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:51:46.937561 3849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.938767 3832 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:51:46.938905 3832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "2183d147333749f5b4bf990fb5904774"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "369657a4ec9babc9c3bb7f0bba5c9f44"
server_key_iv: "48e7bbf9bdc2e3c792abcb5652962a74"
server_key_version: "encryptionkey@0"
I20260809 07:51:46.939450 3832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:51:46.950614 3832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:51:46.953313 3832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:51:46.953498 3832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:51:46.954051 3832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:51:46.954831 3832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:51:46.954900 3832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.954974 3832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:51:46.955020 3832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:51:46.963958 3832 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:33199
I20260809 07:51:46.963970 3971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:33199 every 8 connection(s)
I20260809 07:51:46.964994 3832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:51:46.973299 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 3832
Aug 09 07:51:46 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261906, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:51:46.978833 3682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.966749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:53577 (local address 127.18.71.126:33033)
0809 07:51:46.966881 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:46.966884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:46.968151 (+ 1267us) server_negotiation.cc:407] Connection header received
0809 07:51:46.969121 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:46.969124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:46.969181 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:46.969255 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:46.970365 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.970871 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.971978 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.972194 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.972268 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.975054 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:46.975057 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:46.975059 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:46.975082 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:46.976739 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.977237 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.977240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.977242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.977337 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.977696 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.977699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.977701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.977891 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:46.977990 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:46.978616 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 07:51:46.978719 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:51:46.979727 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.967119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:53577)
0809 07:51:46.968000 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:46.968044 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:46.968856 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:46.969417 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:46.969424 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:46.969765 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:51:46.970243 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.970256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.971014 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.971017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.971873 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:46.971881 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.972316 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.972319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:46.972509 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.973071 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:51:46.973097 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:51:46.974887 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:51:46.976850 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.976856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.976871 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.977134 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.977434 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:51:46.977437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:51:46.977438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:51:46.977583 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:51:46.978013 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:51:46.978018 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:51:46.978338 (+ 320us) client_negotiation.cc:769] Sending connection context
0809 07:51:46.978558 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 07:51:46.978916 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"thread_start_us":544,"threads_started":1}
I20260809 07:51:46.981187 3973 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33033
I20260809 07:51:46.981436 3973 heartbeater.cc:464] Registering TS with master...
I20260809 07:51:46.981927 3973 heartbeater.cc:511] Master 127.18.71.126:33033 requested a full tablet report, sending...
I20260809 07:51:46.982517 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.974142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.18.71.126:33033)
0809 07:51:46.974440 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:46.974444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:46.974461 (+ 17us) server_negotiation.cc:407] Connection header received
0809 07:51:46.974576 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:46.974579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:46.974627 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:46.974717 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:46.975468 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.976284 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.977302 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.977585 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.977690 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.979142 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:46.979145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:46.979148 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:46.979180 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:46.980835 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.981312 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.981316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.981318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.981401 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.981697 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.981701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.981703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.981875 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:46.982114 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:46.982229 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 07:51:46.982347 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":90,"threads_started":1}
I20260809 07:51:46.983222 3472 ts_manager.cc:194] Registered new tserver with Master: 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199)
I20260809 07:51:46.983923 3472 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:53577
I20260809 07:51:46.984483 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:51:46.988375 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.985141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.18.71.126:33033)
0809 07:51:46.985266 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:46.985271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:46.985480 (+ 209us) server_negotiation.cc:407] Connection header received
0809 07:51:46.985637 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:46.985639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:46.985726 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:46.985806 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:51:46.986047 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.986526 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.987370 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.987580 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.987656 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.987945 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:51:46.988062 (+ 117us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 07:51:46.988259 (+ 197us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56234: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 07:51:46.988444 3977 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56234: token expired
W20260809 07:51:46.988837 3394 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.18.71.126:33033)
I20260809 07:51:46.995800 3977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:46.989221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.18.71.126:33033)
0809 07:51:46.989332 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:46.989336 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:46.989356 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:51:46.989442 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:46.989444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:46.989478 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:46.989541 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:51:46.990082 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.990529 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.991275 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:46.991475 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:46.991567 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:46.992876 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:51:46.992880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:51:46.992882 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:51:46.992909 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:51:46.994188 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.994618 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.994621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.994623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.994702 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:51:46.994967 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:51:46.994970 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:51:46.994972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:51:46.995141 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:51:46.995272 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:46.995496 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 07:51:46.995624 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:51:46.998248 3471 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:51:47.001518 3471 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:51:47.009357 3604 tablet_service.cc:1467] Processing CreateTablet for tablet 703f0fd23deb4ad78ac4d6000944c9d3 (DEFAULT_TABLE table=test-table [id=dfcdbc98cb2c439782b1bcf47216c7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:47.010073 3604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703f0fd23deb4ad78ac4d6000944c9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:47.013092 3559 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap starting.
I20260809 07:51:47.013751 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.008169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33199 (local address 127.0.0.1:59288)
0809 07:51:47.008295 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:47.008310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:47.008392 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:47.009464 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:47.009467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:47.009485 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:47.009609 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.009620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.010979 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.010983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.011904 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.011913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.013313 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.013316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.013439 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.013498 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 07:51:47.013553 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 07:51:47.013648 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 07:51:47.013998 3559 tablet_bootstrap.cc:654] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:47.014537 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.008386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.18.71.67:33199)
0809 07:51:47.008916 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:47.008936 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:47.008961 (+ 25us) server_negotiation.cc:407] Connection header received
0809 07:51:47.009054 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:47.009061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:47.009308 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:47.009487 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:47.009749 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.010868 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.012060 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.012943 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.014045 (+ 1102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.014222 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:47.014281 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 07:51:47.014388 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":185,"threads_started":1}
I20260809 07:51:47.014582 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.008436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35299 (local address 127.0.0.1:55996)
0809 07:51:47.008725 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:47.008741 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:47.008822 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:47.009753 (+ 931us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:47.009759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:47.009777 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:47.009898 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.009905 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.011661 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.011664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.012566 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.012581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.014180 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.014184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.014296 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.014319 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:51:47.014385 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 07:51:47.014448 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":120,"threads_started":1}
I20260809 07:51:47.014693 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.008899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55996 (local address 127.18.71.66:35299)
0809 07:51:47.009273 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:47.009285 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:47.009311 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:51:47.009390 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:47.009397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:47.009587 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:47.010061 (+ 474us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:47.010454 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.011534 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.012827 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.014005 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.014184 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.014354 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:47.014424 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 07:51:47.014523 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":144,"threads_started":1}
I20260809 07:51:47.015906 3559 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: No bootstrap required, opened a new log
I20260809 07:51:47.016007 3559 ts_tablet_manager.cc:1397] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 07:51:47.016523 3559 raft_consensus.cc:348] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.016659 3559 raft_consensus.cc:374] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:47.016698 3559 raft_consensus.cc:729] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02613b4ab26b41199ee3a800c533a8a1, State: Initialized, Role: FOLLOWER
I20260809 07:51:47.016856 3559 consensus_queue.cc:260] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.017215 3559 ts_tablet_manager.cc:1428] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 07:51:47.017683 3899 tablet_service.cc:1467] Processing CreateTablet for tablet 703f0fd23deb4ad78ac4d6000944c9d3 (DEFAULT_TABLE table=test-table [id=dfcdbc98cb2c439782b1bcf47216c7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:47.017684 3756 tablet_service.cc:1467] Processing CreateTablet for tablet 703f0fd23deb4ad78ac4d6000944c9d3 (DEFAULT_TABLE table=test-table [id=dfcdbc98cb2c439782b1bcf47216c7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:51:47.018785 3756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703f0fd23deb4ad78ac4d6000944c9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:47.018719 3899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 703f0fd23deb4ad78ac4d6000944c9d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:51:47.025710 3988 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Bootstrap starting.
I20260809 07:51:47.025712 3989 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Bootstrap starting.
I20260809 07:51:47.027864 3988 tablet_bootstrap.cc:654] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:47.028255 3989 tablet_bootstrap.cc:654] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Neither blocks nor log segments found. Creating new log.
I20260809 07:51:47.028972 3989 log.cc:824] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:47.028973 3988 log.cc:824] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Log is configured to *not* fsync() on all Append() calls
I20260809 07:51:47.030684 3989 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: No bootstrap required, opened a new log
I20260809 07:51:47.030697 3988 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: No bootstrap required, opened a new log
I20260809 07:51:47.030872 3989 ts_tablet_manager.cc:1397] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 07:51:47.030881 3988 ts_tablet_manager.cc:1397] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 07:51:47.034021 3989 raft_consensus.cc:348] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.034088 3988 raft_consensus.cc:348] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.034288 3989 raft_consensus.cc:374] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:47.034286 3988 raft_consensus.cc:374] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:51:47.034371 3988 raft_consensus.cc:729] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2183d147333749f5b4bf990fb5904774, State: Initialized, Role: FOLLOWER
I20260809 07:51:47.034373 3989 raft_consensus.cc:729] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8040b2316874780bd02b96d4413acf3, State: Initialized, Role: FOLLOWER
I20260809 07:51:47.034830 3989 consensus_queue.cc:260] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.034830 3988 consensus_queue.cc:260] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.035535 3829 heartbeater.cc:503] Master 127.18.71.126:33033 was elected leader, sending a full tablet report...
I20260809 07:51:47.035759 3973 heartbeater.cc:503] Master 127.18.71.126:33033 was elected leader, sending a full tablet report...
I20260809 07:51:47.035830 3989 ts_tablet_manager.cc:1428] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 07:51:47.036762 3988 ts_tablet_manager.cc:1428] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 07:51:47.079501 3681 tablet.cc:2369] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:51:47.087383 3994 raft_consensus.cc:482] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:51:47.087762 3994 raft_consensus.cc:504] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.088925 3994 leader_election.cc:288] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299), 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467)
I20260809 07:51:47.093863 3688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.089263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:53493 (local address 127.18.71.65:40467)
0809 07:51:47.089392 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:47.089396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:47.089679 (+ 283us) server_negotiation.cc:407] Connection header received
0809 07:51:47.089842 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:47.089845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:47.089898 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:47.090005 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:47.090244 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.090981 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.092388 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.093127 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.093234 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.093368 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:47.093661 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 07:51:47.093733 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:51:47.093861 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.089249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:40947 (local address 127.18.71.66:35299)
0809 07:51:47.089378 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:47.089383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:47.089398 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:51:47.089544 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:47.089547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:47.089594 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:47.089659 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:51:47.090025 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.090811 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.092388 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.093079 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.093221 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.093274 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:47.093662 (+ 388us) server_negotiation.cc:299] Negotiation successful
0809 07:51:47.093733 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:51:47.093935 3975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.089125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35299 (local address 127.18.71.67:40947)
0809 07:51:47.089323 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:47.089338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:47.089428 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:47.089739 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:47.089744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:47.089763 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:47.089893 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.089898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.090926 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.090928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.092262 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.092270 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.093272 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.093277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.093488 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.093530 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 07:51:47.093633 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 07:51:47.093747 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":98}
I20260809 07:51:47.094156 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.089155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40467 (local address 127.18.71.67:53493)
0809 07:51:47.089587 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:47.089597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 07:51:47.089722 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:51:47.090029 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:51:47.090032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:51:47.090048 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:51:47.090135 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.090139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.091069 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.091072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.092258 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:51:47.092270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.093272 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.093277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:51:47.093483 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.093543 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 07:51:47.093633 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 07:51:47.093747 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":90,"thread_start_us":135,"threads_started":1}
I20260809 07:51:47.094817 3778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703f0fd23deb4ad78ac4d6000944c9d3" candidate_uuid: "2183d147333749f5b4bf990fb5904774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8040b2316874780bd02b96d4413acf3" is_pre_election: true
I20260809 07:51:47.094800 3627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703f0fd23deb4ad78ac4d6000944c9d3" candidate_uuid: "2183d147333749f5b4bf990fb5904774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02613b4ab26b41199ee3a800c533a8a1" is_pre_election: true
I20260809 07:51:47.095111 3778 raft_consensus.cc:2454] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2183d147333749f5b4bf990fb5904774 in term 0.
I20260809 07:51:47.095111 3627 raft_consensus.cc:2454] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2183d147333749f5b4bf990fb5904774 in term 0.
I20260809 07:51:47.095584 3851 leader_election.cc:302] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02613b4ab26b41199ee3a800c533a8a1, 2183d147333749f5b4bf990fb5904774; no voters:
I20260809 07:51:47.095829 3994 raft_consensus.cc:2781] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:51:47.095921 3994 raft_consensus.cc:482] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:51:47.095975 3994 raft_consensus.cc:3037] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:47.097158 3994 raft_consensus.cc:504] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.097496 3994 leader_election.cc:288] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [CANDIDATE]: Term 1 election: Requested vote from peers b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299), 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467)
I20260809 07:51:47.097844 3778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703f0fd23deb4ad78ac4d6000944c9d3" candidate_uuid: "2183d147333749f5b4bf990fb5904774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8040b2316874780bd02b96d4413acf3"
I20260809 07:51:47.097888 3627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "703f0fd23deb4ad78ac4d6000944c9d3" candidate_uuid: "2183d147333749f5b4bf990fb5904774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02613b4ab26b41199ee3a800c533a8a1"
I20260809 07:51:47.098004 3627 raft_consensus.cc:3037] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:47.097990 3778 raft_consensus.cc:3037] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:51:47.099327 3627 raft_consensus.cc:2454] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2183d147333749f5b4bf990fb5904774 in term 1.
I20260809 07:51:47.099327 3778 raft_consensus.cc:2454] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2183d147333749f5b4bf990fb5904774 in term 1.
I20260809 07:51:47.099619 3851 leader_election.cc:302] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02613b4ab26b41199ee3a800c533a8a1, 2183d147333749f5b4bf990fb5904774; no voters:
I20260809 07:51:47.099794 3994 raft_consensus.cc:2781] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:51:47.100037 3994 raft_consensus.cc:686] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 1 LEADER]: Becoming Leader. State: Replica: 2183d147333749f5b4bf990fb5904774, State: Running, Role: LEADER
I20260809 07:51:47.100382 3994 consensus_queue.cc:237] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:51:47.103547 3471 catalog_manager.cc:5495] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2183d147333749f5b4bf990fb5904774 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "2183d147333749f5b4bf990fb5904774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } health_report { overall_health: UNKNOWN } } }
I20260809 07:51:47.114265 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:51:47.110994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59304 (local address 127.18.71.67:33199)
0809 07:51:47.111108 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 07:51:47.111112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:51:47.111124 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:51:47.111259 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:51:47.111262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:51:47.111311 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:51:47.111390 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:51:47.111603 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.112073 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.112908 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:51:47.113105 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:51:47.113180 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:51:47.113515 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:51:47.113917 (+ 402us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:51:47.114003 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 07:51:47.114104 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 07:51:47.114150 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 07:51:47.120165 3778 raft_consensus.cc:1261] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 1 FOLLOWER]: Refusing update from remote peer 2183d147333749f5b4bf990fb5904774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:47.120136 3627 raft_consensus.cc:1261] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Refusing update from remote peer 2183d147333749f5b4bf990fb5904774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:51:47.120954 3994 consensus_queue.cc:1035] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:47.121218 3996 consensus_queue.cc:1035] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:51:47.126734 4002 mvcc.cc:205] Tried to move back new op lower bound from 7316528771556384768 to 7316528771485704192. Current Snapshot: MvccSnapshot[applied={T|T < 7316528771556384768}]
I20260809 07:51:47.128688 4001 mvcc.cc:205] Tried to move back new op lower bound from 7316528771556384768 to 7316528771485704192. Current Snapshot: MvccSnapshot[applied={T|T < 7316528771556384768}]
I20260809 07:51:47.130072 4003 mvcc.cc:205] Tried to move back new op lower bound from 7316528771556384768 to 7316528771485704192. Current Snapshot: MvccSnapshot[applied={T|T < 7316528771556384768}]
I20260809 07:51:47.137015 3471 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:47.137209 3471 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:51:47.139505 3471 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Sending DeleteTablet for 3 replicas of tablet 703f0fd23deb4ad78ac4d6000944c9d3
I20260809 07:51:47.140208 3899 tablet_service.cc:1514] Processing DeleteTablet for tablet 703f0fd23deb4ad78ac4d6000944c9d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:47 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:59288
I20260809 07:51:47.140256 3756 tablet_service.cc:1514] Processing DeleteTablet for tablet 703f0fd23deb4ad78ac4d6000944c9d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:47 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:55996
I20260809 07:51:47.140478 3604 tablet_service.cc:1514] Processing DeleteTablet for tablet 703f0fd23deb4ad78ac4d6000944c9d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:51:47 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:45748
I20260809 07:51:47.140852 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 3513
I20260809 07:51:47.140864 4012 tablet_replica.cc:330] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: stopping tablet replica
I20260809 07:51:47.143257 4011 tablet_replica.cc:330] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: stopping tablet replica
I20260809 07:51:47.143610 4011 raft_consensus.cc:2229] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:51:47.143966 4011 raft_consensus.cc:2258] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:47.145625 4011 ts_tablet_manager.cc:1905] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:51:47.148931 4013 tablet_replica.cc:330] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: stopping tablet replica
I20260809 07:51:47.149310 4013 raft_consensus.cc:2229] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:51:47.149365 4011 ts_tablet_manager.cc:1918] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:47.149477 4011 log.cc:1196] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/703f0fd23deb4ad78ac4d6000944c9d3
I20260809 07:51:47.149847 4011 ts_tablet_manager.cc:1939] T 703f0fd23deb4ad78ac4d6000944c9d3 P b8040b2316874780bd02b96d4413acf3: Deleting consensus metadata
I20260809 07:51:47.149957 4013 raft_consensus.cc:2258] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:51:47.151242 3458 catalog_manager.cc:4841] TS b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299): tablet 703f0fd23deb4ad78ac4d6000944c9d3 (table test-table [id=dfcdbc98cb2c439782b1bcf47216c7b4]) successfully deleted
I20260809 07:51:47.151803 4013 ts_tablet_manager.cc:1905] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 07:51:47.151846 3455 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:40467 (error 108)
W20260809 07:51:47.153129 3983 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:51:47.152817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40467 (local address 127.0.0.1:45754)
0809 07:51:47.152951 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 07:51:47.153030 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40467: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 07:51:47.153277 3455 catalog_manager.cc:4584] TS 02613b4ab26b41199ee3a800c533a8a1 (127.18.71.65:40467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 703f0fd23deb4ad78ac4d6000944c9d3: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40467: connect: Connection reset by peer (error 104)
I20260809 07:51:47.154088 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 3684
I20260809 07:51:47.162098 4013 ts_tablet_manager.cc:1918] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:51:47.162216 4013 log.cc:1196] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/703f0fd23deb4ad78ac4d6000944c9d3
I20260809 07:51:47.162576 4013 ts_tablet_manager.cc:1939] T 703f0fd23deb4ad78ac4d6000944c9d3 P 2183d147333749f5b4bf990fb5904774: Deleting consensus metadata
I20260809 07:51:47.163916 3457 catalog_manager.cc:4841] TS 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199): tablet 703f0fd23deb4ad78ac4d6000944c9d3 (table test-table [id=dfcdbc98cb2c439782b1bcf47216c7b4]) successfully deleted
I20260809 07:51:47.167063 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 3832
I20260809 07:51:47.175164 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 3437
I20260809 07:52:07.183477 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33033
--webserver_interface=127.18.71.126
--webserver_port=41697
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:07.296849 4015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:07.297093 4015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:07.297139 4015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:07.301075 4015 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 07:52:07.301131 4015 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 07:52:07.301146 4015 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 07:52:07.301160 4015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:07.301213 4015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:07.301247 4015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:07.301280 4015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:07.301313 4015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:07.306452 4015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=41697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:07.307453 4015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:07.308516 4015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:07.316790 4023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:07.316949 4015 server_base.cc:1034] running on GCE node
W20260809 07:52:07.316799 4021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:07.316809 4020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:07.317646 4015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:07.318806 4015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:07.320032 4015 hybrid_clock.cc:648] HybridClock initialized: now 1786261927319979 us; error 42 us; skew 500 ppm
Aug 09 07:52:07 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:07.322847 4015 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:07.323704 4015 webserver.cc:460] Webserver started at http://127.18.71.126:41697/ using document root <none> and password file <none>
I20260809 07:52:07.324271 4015 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:07.324365 4015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:07.328101 4015 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:07.330258 4030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:07.331480 4015 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:07.331627 4015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "9fc6cb9315d749fea595e60f6a1e39eb"
format_stamp: "Formatted at 2026-08-09 07:51:24 on dist-test-slave-10pc"
server_key: "35446d2a95cbdd20b15df7c687d49b6d"
server_key_iv: "2bbf1001ed7a8fcb95ca6e34cf04a8e4"
server_key_version: "encryptionkey@0"
I20260809 07:52:07.332194 4015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:07.345898 4015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:07.348914 4015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:07.349123 4015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:07.355923 4015 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33033
I20260809 07:52:07.356240 4085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33033 every 8 connection(s)
I20260809 07:52:07.357052 4015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:07.360903 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 4015
I20260809 07:52:07.361426 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:40467
--local_ip_for_outbound_sockets=127.18.71.65
--tserver_master_addrs=127.18.71.126:33033
--webserver_port=42941
--webserver_interface=127.18.71.65
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:07.361756 4086 sys_catalog.cc:263] Verifying existing consensus state
I20260809 07:52:07.364147 4086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Bootstrap starting.
I20260809 07:52:07.381871 4086 log.cc:824] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:07.393471 4086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:07.393870 4086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: Bootstrap complete.
I20260809 07:52:07.398255 4086 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:52:07.398660 4086 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fc6cb9315d749fea595e60f6a1e39eb, State: Initialized, Role: FOLLOWER
I20260809 07:52:07.399155 4086 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:52:07.399274 4086 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:07.399364 4086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:07.399483 4086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 2 FOLLOWER]: Advancing to term 3
I20260809 07:52:07.401480 4086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:52:07.401830 4086 leader_election.cc:302] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fc6cb9315d749fea595e60f6a1e39eb; no voters:
I20260809 07:52:07.402179 4086 leader_election.cc:288] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 07:52:07.402293 4090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 07:52:07.402508 4090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [term 3 LEADER]: Becoming Leader. State: Replica: 9fc6cb9315d749fea595e60f6a1e39eb, State: Running, Role: LEADER
I20260809 07:52:07.402876 4090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } }
I20260809 07:52:07.403471 4086 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:07.404371 4092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fc6cb9315d749fea595e60f6a1e39eb. Latest consensus state: current_term: 3 leader_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } } }
I20260809 07:52:07.404426 4091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fc6cb9315d749fea595e60f6a1e39eb" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33033 } } }
I20260809 07:52:07.404585 4091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:07.404529 4092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:07.409526 4107 catalog_manager.cc:1259] Loaded cluster ID: 335f9420fc7a4bc196888e0d3f8091c8
I20260809 07:52:07.409590 4107 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: loading cluster ID for follower catalog manager: success
I20260809 07:52:07.412038 4107 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9fc6cb9315d749fea595e60f6a1e39eb: acquiring CA information for follower catalog manager: success
F20260809 07:52:07.413228 4107 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdf0f4a79ad google::LogMessage::Fail() at ??:0
@ 0x7fdf0f4aa88a google::LogMessage::SendToLog() at ??:0
@ 0x7fdf0f4a74de google::LogMessage::Flush() at ??:0
@ 0x7fdf0f4a8a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdf1216b18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fdf19e10989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7fdf19ded023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7fdf19ded54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7fdf19de6652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7fdf19de59e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7fdf19e2c541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fdf116ea512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdf12d2e6db start_thread at ??:0
@ 0x7fdf0dfa561f clone at ??:0
W20260809 07:52:07.503580 4088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:07.503829 4088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:07.503871 4088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:07.507874 4088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:07.507932 4088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:07.508021 4088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:52:07.513609 4088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:40467
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=42941
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:07.514791 4088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:07.515935 4088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:07.526404 4131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:07.526423 4132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:07.526402 4135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:07.527355 4088 server_base.cc:1034] running on GCE node
I20260809 07:52:07.527787 4088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:07.528522 4088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:07.529716 4088 hybrid_clock.cc:648] HybridClock initialized: now 1786261927529687 us; error 44 us; skew 500 ppm
Aug 09 07:52:07 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:07.532878 4088 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:52:07.533957 4088 webserver.cc:460] Webserver started at http://127.18.71.65:42941/ using document root <none> and password file <none>
I20260809 07:52:07.534709 4088 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:07.534855 4088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:07.539985 4088 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:07.542897 4143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:07.544494 4088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:07.544620 4088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "02613b4ab26b41199ee3a800c533a8a1"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "c5b6a8ec353d6e6336754d64955f6ab2"
server_key_iv: "81af6814ad61a570d88016466d9f75b5"
server_key_version: "encryptionkey@0"
I20260809 07:52:07.545397 4088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:07.575196 4088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:07.579399 4088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:07.579620 4088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:07.580400 4088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:07.581902 4151 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 07:52:07.586231 4088 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 07:52:07.586313 4088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 07:52:07.586378 4088 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 07:52:07.588737 4088 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 07:52:07.588799 4088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:07.589172 4151 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap starting.
I20260809 07:52:07.600124 4088 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:40467
I20260809 07:52:07.600260 4265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:40467 every 8 connection(s)
I20260809 07:52:07.601485 4088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:07.609477 4151 log.cc:824] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:07.609974 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 4088
I20260809 07:52:07.610443 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:35299
--local_ip_for_outbound_sockets=127.18.71.66
--tserver_master_addrs=127.18.71.126:33033
--webserver_port=41169
--webserver_interface=127.18.71.66
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:52:07.618721 4151 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 07:52:07.619161 4151 tablet_bootstrap.cc:492] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Bootstrap complete.
I20260809 07:52:07.620718 4151 ts_tablet_manager.cc:1397] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.020s
I20260809 07:52:07.624326 4151 raft_consensus.cc:348] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:52:07.625408 4151 raft_consensus.cc:729] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02613b4ab26b41199ee3a800c533a8a1, State: Initialized, Role: FOLLOWER
I20260809 07:52:07.626134 4151 consensus_queue.cc:260] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:52:07.627243 4151 ts_tablet_manager.cc:1428] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260809 07:52:07.634724 4268 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:52:07.603852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:50365)
0809 07:52:07.604507 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:07.604579 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:07.605710 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:07.633907 (+ 28197us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":171,"threads_started":1}
W20260809 07:52:07.635324 4266 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:33033 (0 consecutive failures): Network error: Failed to ping master at 127.18.71.126:33033: Client connection negotiation failed: client connection to 127.18.71.126:33033: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 07:52:07.636062 4268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:07.635817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:38243)
0809 07:52:07.635940 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:07.636012 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 07:52:07.636822 4268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:07.636605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:36515)
0809 07:52:07.636720 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:07.636766 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 07:52:07.637049 4266 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 07:52:07.730448 4270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:07.730690 4270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:07.730729 4270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:07.734740 4270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:07.734794 4270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:07.734884 4270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:52:07.740058 4270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:35299
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=41169
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:07.741117 4270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:07.742087 4270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:07.751597 4277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:07.751569 4280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:07.751569 4278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:07.751971 4270 server_base.cc:1034] running on GCE node
I20260809 07:52:07.752405 4270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:07.753094 4270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:07.754310 4270 hybrid_clock.cc:648] HybridClock initialized: now 1786261927754287 us; error 32 us; skew 500 ppm
Aug 09 07:52:07 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:07.757113 4270 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:52:07.757954 4270 webserver.cc:460] Webserver started at http://127.18.71.66:41169/ using document root <none> and password file <none>
I20260809 07:52:07.758514 4270 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:07.758620 4270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:07.762512 4270 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 07:52:07.764664 4290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:07.765908 4270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:52:07.766052 4270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b8040b2316874780bd02b96d4413acf3"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "9376924841ae7472fb61cf7d89dd1a5a"
server_key_iv: "141c395cb3241e3b24247cb7da96d887"
server_key_version: "encryptionkey@0"
I20260809 07:52:07.766625 4270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:07.782871 4270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:07.786271 4270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:07.786481 4270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:07.787102 4270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:07.787958 4270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:07.788031 4270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:07.788100 4270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:07.788151 4270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:07.797694 4270 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:35299
I20260809 07:52:07.797770 4407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:35299 every 8 connection(s)
I20260809 07:52:07.798753 4270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:07.802595 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:07.800894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:32945)
0809 07:52:07.801330 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:07.801435 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":113,"threads_started":1}
W20260809 07:52:07.803073 4408 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:33033 (0 consecutive failures): Network error: Failed to ping master at 127.18.71.126:33033: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
I20260809 07:52:07.803644 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:07.803434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:47187)
0809 07:52:07.803533 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:07.803593 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 07:52:07.804206 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:07.804031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:39967)
0809 07:52:07.804124 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:07.804164 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 07:52:07.804390 4408 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 07:52:07.808579 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 4270
I20260809 07:52:07.809012 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:33199
--local_ip_for_outbound_sockets=127.18.71.67
--tserver_master_addrs=127.18.71.126:33033
--webserver_port=46205
--webserver_interface=127.18.71.67
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:07.925729 4411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:07.925983 4411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:07.926026 4411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:07.930135 4411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:07.930199 4411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:07.930298 4411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:52:07.935770 4411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:33199
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=46205
--tserver_master_addrs=127.18.71.126:33033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:07.936957 4411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:07.937999 4411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:07.947808 4420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:07.948040 4411 server_base.cc:1034] running on GCE node
W20260809 07:52:07.947858 4416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:07.947813 4418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:07.948729 4411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:07.949404 4411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:07.950583 4411 hybrid_clock.cc:648] HybridClock initialized: now 1786261927950562 us; error 33 us; skew 500 ppm
Aug 09 07:52:07 dist-test-slave-10pc krb5kdc[2842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:07.953851 4411 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:52:07.954835 4411 webserver.cc:460] Webserver started at http://127.18.71.67:46205/ using document root <none> and password file <none>
I20260809 07:52:07.955417 4411 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:07.955483 4411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:07.959759 4411 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:52:07.962034 4427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:07.963368 4411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:52:07.963474 4411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "2183d147333749f5b4bf990fb5904774"
format_stamp: "Formatted at 2026-08-09 07:51:25 on dist-test-slave-10pc"
server_key: "369657a4ec9babc9c3bb7f0bba5c9f44"
server_key_iv: "48e7bbf9bdc2e3c792abcb5652962a74"
server_key_version: "encryptionkey@0"
I20260809 07:52:07.964048 4411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:07.988967 4411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:07.992564 4411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:07.992748 4411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:07.993513 4411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:07.994480 4411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:07.994546 4411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:07.994627 4411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:07.994671 4411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:08.005142 4411 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:33199
I20260809 07:52:08.005165 4550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:33199 every 8 connection(s)
I20260809 07:52:08.006322 4411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:08.007136 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 4411
I20260809 07:52:08.010842 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:08.009054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:54203)
0809 07:52:08.009538 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:08.009662 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":142,"threads_started":1}
W20260809 07:52:08.011313 4551 heartbeater.cc:644] Failed to heartbeat to 127.18.71.126:33033 (0 consecutive failures): Network error: Failed to ping master at 127.18.71.126:33033: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
I20260809 07:52:08.011956 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:08.011732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:54887)
0809 07:52:08.011845 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:08.011902 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 07:52:08.012596 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:08.012364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:58723)
0809 07:52:08.012494 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:08.012546 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 07:52:08.012799 4551 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 07:52:08.638098 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:08.637644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:44311)
0809 07:52:08.637927 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:08.637992 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":71,"threads_started":1}
I20260809 07:52:08.805248 4556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:08.804845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:41209)
0809 07:52:08.805097 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:08.805151 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":69,"threads_started":1}
I20260809 07:52:09.013726 4557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:09.013343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:36857)
0809 07:52:09.013573 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:09.013632 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260809 07:52:09.228350 4558 raft_consensus.cc:482] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:09.228644 4558 raft_consensus.cc:504] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:52:09.229748 4558 leader_election.cc:288] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199), b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299)
Aug 09 07:52:09 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.66@KRBTEST.COM
Aug 09 07:52:09 dist-test-slave-10pc krb5kdc[2842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261927, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.67@KRBTEST.COM
I20260809 07:52:09.244469 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:09.229979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33199 (local address 127.18.71.65:46569)
0809 07:52:09.230361 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:09.230389 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:09.230518 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:09.232075 (+ 1557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:09.232081 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:09.232667 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:09.233182 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:09.233203 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.234655 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.234658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:09.235487 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:09.235497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.236247 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.236251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:09.236424 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:09.237298 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:09.237319 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:09.239254 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:09.241490 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:09.241497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:09.241502 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:09.241855 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:09.242250 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:09.242253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:09.242255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:09.242465 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:09.243224 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:09.243231 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:09.243599 (+ 368us) client_negotiation.cc:769] Sending connection context
0809 07:52:09.243817 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 07:52:09.244214 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":213,"spinlock_wait_cycles":42112,"thread_start_us":90,"threads_started":1}
I20260809 07:52:09.244506 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:09.230168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:46569 (local address 127.18.71.67:33199)
0809 07:52:09.230554 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:09.230577 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:09.230613 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:52:09.231347 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:09.231354 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:09.231698 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:09.231951 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:09.233497 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.234506 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.235665 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.236120 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.236304 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:09.239442 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:09.239449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:09.239465 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:09.239504 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:09.241390 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:09.241978 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:09.241984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:09.241990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:09.242147 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:09.242575 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:09.242579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:09.242581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:09.243112 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:09.243237 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:09.243958 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 07:52:09.244281 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":120,"threads_started":1}
I20260809 07:52:09.244470 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:09.229979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:35299 (local address 127.18.71.65:39033)
0809 07:52:09.230467 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:09.230479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:09.230537 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:09.232074 (+ 1537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:09.232080 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:09.232667 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:09.233181 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:09.233200 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.234590 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.234595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:09.235485 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:09.235494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.236215 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.236219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:09.236497 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:09.237218 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:09.237255 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:09.238914 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:09.241389 (+ 2475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:09.241395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:09.241409 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:09.241855 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:09.242250 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:09.242253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:09.242255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:09.242465 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:09.243224 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:09.243231 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:09.243600 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 07:52:09.243817 (+ 217us) client_negotiation.cc:240] Negotiation successful
0809 07:52:09.244269 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":329,"spinlock_wait_cycles":42496,"thread_start_us":70,"threads_started":1}
I20260809 07:52:09.245188 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:09.230154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:39033 (local address 127.18.71.66:35299)
0809 07:52:09.230763 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:09.230784 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:09.230816 (+ 32us) server_negotiation.cc:407] Connection header received
0809 07:52:09.231487 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:09.231492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:09.231785 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:09.232089 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:09.233369 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.234428 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.235613 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:09.236078 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:09.236267 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:09.239140 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:09.239144 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:09.239156 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:09.239183 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:09.241261 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:09.241979 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:09.241984 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:09.241990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:09.242140 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:09.242595 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:09.242598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:09.242599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:09.243097 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:09.243237 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:09.244705 (+ 1468us) server_negotiation.cc:299] Negotiation successful
0809 07:52:09.244972 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":73,"threads_started":1}
W20260809 07:52:09.247438 4147 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
W20260809 07:52:09.247650 4148 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
I20260809 07:52:09.247740 4148 leader_election.cc:302] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02613b4ab26b41199ee3a800c533a8a1; no voters: 2183d147333749f5b4bf990fb5904774, b8040b2316874780bd02b96d4413acf3
I20260809 07:52:09.248044 4558 raft_consensus.cc:2726] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:52:09.639101 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:09.638830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:53067)
0809 07:52:09.638959 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:09.639015 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 07:52:09.806449 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:09.805977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:37583)
0809 07:52:09.806283 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:09.806349 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":111,"threads_started":1}
I20260809 07:52:10.014838 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:10.014436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:55273)
0809 07:52:10.014696 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:10.014753 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":91,"threads_started":1}
I20260809 07:52:10.640537 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:10.639950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:48177)
0809 07:52:10.640305 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:10.640392 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":134,"threads_started":1}
I20260809 07:52:10.807497 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:10.807122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:42145)
0809 07:52:10.807359 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:10.807412 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":74,"threads_started":1}
I20260809 07:52:11.015977 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:11.015576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:52631)
0809 07:52:11.015835 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:11.015891 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":92,"threads_started":1}
I20260809 07:52:11.414175 4569 raft_consensus.cc:482] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:11.414340 4569 raft_consensus.cc:504] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:52:11.414805 4569 leader_election.cc:288] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199), b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299)
W20260809 07:52:11.415719 4147 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
W20260809 07:52:11.415827 4148 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
I20260809 07:52:11.415894 4148 leader_election.cc:302] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02613b4ab26b41199ee3a800c533a8a1; no voters: 2183d147333749f5b4bf990fb5904774, b8040b2316874780bd02b96d4413acf3
I20260809 07:52:11.416046 4569 raft_consensus.cc:2726] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:52:11.641696 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:11.641236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:51111)
0809 07:52:11.641515 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:11.641569 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":75,"threads_started":1}
I20260809 07:52:11.808702 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:11.808238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:38919)
0809 07:52:11.808554 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:11.808607 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260809 07:52:12.017302 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:12.016850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:44265)
0809 07:52:12.017137 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:12.017203 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260809 07:52:12.643052 4574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:12.642637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:40401)
0809 07:52:12.642907 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:12.642958 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":73,"threads_started":1}
I20260809 07:52:12.810200 4575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:12.809623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:36307)
0809 07:52:12.809971 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:12.810052 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":91,"threads_started":1}
I20260809 07:52:12.977440 4576 raft_consensus.cc:482] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:12.977586 4576 raft_consensus.cc:504] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:52:12.978089 4576 leader_election.cc:288] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199), b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299)
W20260809 07:52:12.978858 4147 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
W20260809 07:52:12.978995 4148 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
I20260809 07:52:12.979089 4148 leader_election.cc:302] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02613b4ab26b41199ee3a800c533a8a1; no voters: 2183d147333749f5b4bf990fb5904774, b8040b2316874780bd02b96d4413acf3
I20260809 07:52:12.979254 4576 raft_consensus.cc:2726] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:52:13.018636 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:13.018169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:53049)
0809 07:52:13.018469 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:13.018542 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":101,"threads_started":1}
I20260809 07:52:13.644202 4579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:13.643805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:33579)
0809 07:52:13.644048 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:13.644103 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260809 07:52:13.811409 4580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:13.811024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:52471)
0809 07:52:13.811268 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:13.811322 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":86,"threads_started":1}
I20260809 07:52:14.020282 4581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:14.019662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:49957)
0809 07:52:14.020045 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:14.020142 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":142,"threads_started":1}
I20260809 07:52:14.645448 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:14.645002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:33197)
0809 07:52:14.645291 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:14.645349 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":102,"threads_started":1}
I20260809 07:52:14.812665 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:14.812166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:42567)
0809 07:52:14.812521 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:14.812572 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":92,"threads_started":1}
I20260809 07:52:15.021633 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:15.021131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:58743)
0809 07:52:15.021479 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:15.021540 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":96,"threads_started":1}
I20260809 07:52:15.646641 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:15.646236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:59771)
0809 07:52:15.646487 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:15.646541 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260809 07:52:15.813925 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:15.813470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:51407)
0809 07:52:15.813769 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:15.813830 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":112,"threads_started":1}
I20260809 07:52:16.022920 4587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:16.022421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:34065)
0809 07:52:16.022686 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:16.022745 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":100,"threads_started":1}
I20260809 07:52:16.647835 4588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:16.647405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:45153)
0809 07:52:16.647677 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:16.647736 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":96,"threads_started":1}
I20260809 07:52:16.815279 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:16.814812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:34593)
0809 07:52:16.815102 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:16.815172 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":112,"threads_started":1}
I20260809 07:52:16.896662 4590 raft_consensus.cc:482] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:16.896801 4590 raft_consensus.cc:504] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2183d147333749f5b4bf990fb5904774" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33199 } } peers { permanent_uuid: "b8040b2316874780bd02b96d4413acf3" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 35299 } } peers { permanent_uuid: "02613b4ab26b41199ee3a800c533a8a1" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40467 } }
I20260809 07:52:16.897313 4590 leader_election.cc:288] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199), b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299)
W20260809 07:52:16.898077 4147 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2183d147333749f5b4bf990fb5904774 (127.18.71.67:33199): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
W20260809 07:52:16.898200 4148 leader_election.cc:341] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b8040b2316874780bd02b96d4413acf3 (127.18.71.66:35299): Not found: Tablet not found: 703f0fd23deb4ad78ac4d6000944c9d3
I20260809 07:52:16.898313 4148 leader_election.cc:302] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02613b4ab26b41199ee3a800c533a8a1; no voters: 2183d147333749f5b4bf990fb5904774, b8040b2316874780bd02b96d4413acf3
I20260809 07:52:16.898448 4590 raft_consensus.cc:2726] T 703f0fd23deb4ad78ac4d6000944c9d3 P 02613b4ab26b41199ee3a800c533a8a1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 07:52:17.024184 4591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:17.023769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:34747)
0809 07:52:17.024034 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:17.024092 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":82,"threads_started":1}
I20260809 07:52:17.649142 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:17.648724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:55875)
0809 07:52:17.648983 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:17.649041 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260809 07:52:17.816709 4594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:17.816249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:47583)
0809 07:52:17.816549 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:17.816615 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":79,"threads_started":1}
I20260809 07:52:18.025544 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:18.025113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:39759)
0809 07:52:18.025385 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:18.025448 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260809 07:52:18.650609 4596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:18.649993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:46757)
0809 07:52:18.650348 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:18.650433 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":92,"threads_started":1}
I20260809 07:52:18.817994 4597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:18.817596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:53353)
0809 07:52:18.817852 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:18.817904 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
I20260809 07:52:19.026722 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:19.026313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:44071)
0809 07:52:19.026571 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:19.026629 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":75,"threads_started":1}
I20260809 07:52:19.651800 4599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:19.651360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:34641)
0809 07:52:19.651649 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:19.651702 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":91,"threads_started":1}
I20260809 07:52:19.819271 4600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:19.818716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:37699)
0809 07:52:19.819119 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:19.819178 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":72,"threads_started":1}
I20260809 07:52:20.027916 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:20.027503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:39619)
0809 07:52:20.027765 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:20.027822 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":97,"threads_started":1}
I20260809 07:52:20.653101 4602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:20.652663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:53825)
0809 07:52:20.652942 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:20.653003 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260809 07:52:20.820617 4603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:20.820146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:34459)
0809 07:52:20.820411 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:20.820522 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260809 07:52:21.029176 4605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:21.028748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:57619)
0809 07:52:21.029015 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:21.029074 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
I20260809 07:52:21.654306 4606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:21.653872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:37175)
0809 07:52:21.654146 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:21.654207 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
I20260809 07:52:21.821707 4607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:21.821316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:45913)
0809 07:52:21.821562 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:21.821615 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":73,"threads_started":1}
I20260809 07:52:22.030366 4608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:22.029949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:46887)
0809 07:52:22.030214 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:22.030272 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
I20260809 07:52:22.655725 4609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:22.655298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.65:38559)
0809 07:52:22.655554 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:22.655612 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":76,"threads_started":1}
I20260809 07:52:22.823179 4610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:22.822744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.66:40991)
0809 07:52:22.822990 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:22.823068 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":69,"threads_started":1}
I20260809 07:52:23.017761 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 4088
I20260809 07:52:23.026324 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 4270
I20260809 07:52:23.031862 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:23.031440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33033 (local address 127.18.71.67:34615)
0809 07:52:23.031708 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:23.031767 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.126:33033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
I20260809 07:52:23.034502 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 4411
2026-08-09T07:52:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59519 ms)
[----------] 30 tests from SecurityITest (247690 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:23 dist-test-slave-10pc krb5kdc[4614](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:23 dist-test-slave-10pc krb5kdc[4614](info): set up 2 sockets
Aug 09 07:52:23 dist-test-slave-10pc krb5kdc[4614](info): commencing operation
krb5kdc: starting...
W20260809 07:52:24.128222 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:24 dist-test-slave-10pc krb5kdc[4614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:24.263964 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:37953
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:37953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:24.377595 4630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:24.377923 4630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:24.377992 4630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:24.382117 4630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:24.382207 4630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:24.382246 4630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:24.382284 4630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:24.382318 4630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:24.387630 4630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:37953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:37953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:24.388804 4630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:24.389871 4630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:24.398801 4635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:24.398828 4638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:24.398891 4630 server_base.cc:1034] running on GCE node
W20260809 07:52:24.398828 4636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:24.399791 4630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:24.400979 4630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:24.402179 4630 hybrid_clock.cc:648] HybridClock initialized: now 1786261944402139 us; error 30 us; skew 500 ppm
Aug 09 07:52:24 dist-test-slave-10pc krb5kdc[4614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:24.405038 4630 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:24.405982 4630 webserver.cc:460] Webserver started at http://127.18.71.126:43365/ using document root <none> and password file <none>
I20260809 07:52:24.406544 4630 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:24.406606 4630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:24.406855 4630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:24.408987 4630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "de4ea8a59b7049c68558a356bda8f6d5"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "b6fac5b88a229505ccd764018fb0d0a4"
server_key_iv: "d036179f7ba56d58b64078bb3f942197"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.409576 4630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "de4ea8a59b7049c68558a356bda8f6d5"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "b6fac5b88a229505ccd764018fb0d0a4"
server_key_iv: "d036179f7ba56d58b64078bb3f942197"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.413357 4630 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:24.415665 4646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.416952 4630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:24.417081 4630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "de4ea8a59b7049c68558a356bda8f6d5"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "b6fac5b88a229505ccd764018fb0d0a4"
server_key_iv: "d036179f7ba56d58b64078bb3f942197"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.417183 4630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:24.426864 4630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:24.429885 4630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:24.430097 4630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:24.437474 4630 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:37953
I20260809 07:52:24.437505 4701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:37953 every 8 connection(s)
I20260809 07:52:24.438417 4630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:24.440366 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 4630
I20260809 07:52:24.440513 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:24.440804 18717 external_mini_cluster.cc:1428] Setting key 9cd0ef92a008bf2fe6fd4e2ba59afa8e
I20260809 07:52:24.441701 4702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:24.448607 4702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5: Bootstrap starting.
Aug 09 07:52:24 dist-test-slave-10pc krb5kdc[4614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:24.451121 4702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:24.452127 4702 log.cc:824] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:24.454371 4702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5: No bootstrap required, opened a new log
I20260809 07:52:24.455826 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:24.442224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37388 (local address 127.18.71.126:37953)
0809 07:52:24.442662 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:24.442683 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:24.442727 (+ 44us) server_negotiation.cc:407] Connection header received
0809 07:52:24.443597 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:24.443622 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:24.444060 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:24.444438 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:24.445185 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:24.446640 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:24.447510 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:24.448209 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:24.448376 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:24.450842 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:24.450848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:24.450864 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:24.450897 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:24.452851 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:24.453305 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:24.453310 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:24.453316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:24.453451 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:24.453737 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:24.453740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:24.453742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:24.454254 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:24.454448 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:24.454799 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 07:52:24.455115 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":122,"threads_started":1}
I20260809 07:52:24.458572 4702 raft_consensus.cc:348] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4ea8a59b7049c68558a356bda8f6d5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37953 } }
I20260809 07:52:24.458815 4702 raft_consensus.cc:374] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:24.458897 4702 raft_consensus.cc:729] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4ea8a59b7049c68558a356bda8f6d5, State: Initialized, Role: FOLLOWER
I20260809 07:52:24.459388 4702 consensus_queue.cc:260] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4ea8a59b7049c68558a356bda8f6d5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37953 } }
I20260809 07:52:24.459559 4702 raft_consensus.cc:388] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:24.459625 4702 raft_consensus.cc:482] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:24.459722 4702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:24.460680 4702 raft_consensus.cc:504] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4ea8a59b7049c68558a356bda8f6d5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37953 } }
I20260809 07:52:24.461021 4702 leader_election.cc:302] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de4ea8a59b7049c68558a356bda8f6d5; no voters:
I20260809 07:52:24.461328 4702 leader_election.cc:288] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:24.461457 4707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:24.461719 4707 raft_consensus.cc:686] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [term 1 LEADER]: Becoming Leader. State: Replica: de4ea8a59b7049c68558a356bda8f6d5, State: Running, Role: LEADER
I20260809 07:52:24.462064 4707 consensus_queue.cc:237] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4ea8a59b7049c68558a356bda8f6d5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37953 } }
I20260809 07:52:24.462646 4702 sys_catalog.cc:564] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:24.463677 4709 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de4ea8a59b7049c68558a356bda8f6d5. Latest consensus state: current_term: 1 leader_uuid: "de4ea8a59b7049c68558a356bda8f6d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4ea8a59b7049c68558a356bda8f6d5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37953 } } }
I20260809 07:52:24.463812 4709 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:24.464124 4708 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de4ea8a59b7049c68558a356bda8f6d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4ea8a59b7049c68558a356bda8f6d5" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37953 } } }
I20260809 07:52:24.464213 4708 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:24.464226 4717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:24.467689 4717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:24.473289 4717 catalog_manager.cc:1347] Generated new cluster ID: 91c90617eeff46c39dd5b33e132ddb96
I20260809 07:52:24.473366 4717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:24.499152 4717 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:24.500124 4717 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:24.506162 4717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5: Generated new TSK 0
I20260809 07:52:24.506981 4717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:52:24.548128 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:37953
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:24.686100 4730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:24.686403 4730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:24.686486 4730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:24.690528 4730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:24.690624 4730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:24.690719 4730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:52:24.695997 4730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:37953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:24.697094 4730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:24.698135 4730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:24.707980 4735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:24.708094 4738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:24.708170 4730 server_base.cc:1034] running on GCE node
W20260809 07:52:24.708176 4736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:24.708701 4730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:24.709343 4730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:24.710534 4730 hybrid_clock.cc:648] HybridClock initialized: now 1786261944710518 us; error 35 us; skew 500 ppm
Aug 09 07:52:24 dist-test-slave-10pc krb5kdc[4614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:24.713356 4730 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:52:24.714190 4730 webserver.cc:460] Webserver started at http://127.18.71.65:34715/ using document root <none> and password file <none>
I20260809 07:52:24.714727 4730 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:24.714787 4730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:24.714957 4730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:24.717087 4730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "a056c81fc8994d61b6420a4588fc7f31"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "75a96fe0b5277aad9c2bf705d313de16"
server_key_iv: "3c659e82ae0fa4a4552fc45126964767"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.717633 4730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "a056c81fc8994d61b6420a4588fc7f31"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "75a96fe0b5277aad9c2bf705d313de16"
server_key_iv: "3c659e82ae0fa4a4552fc45126964767"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.721380 4730 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 07:52:24.723816 4746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.725109 4730 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:24.725242 4730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "a056c81fc8994d61b6420a4588fc7f31"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "75a96fe0b5277aad9c2bf705d313de16"
server_key_iv: "3c659e82ae0fa4a4552fc45126964767"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.725347 4730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:24.754858 4730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:24.758425 4730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:24.758643 4730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:24.759281 4730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:24.760175 4730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:24.760322 4730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.760418 4730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:24.760491 4730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.769763 4730 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:32819
I20260809 07:52:24.769788 4869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:32819 every 8 connection(s)
I20260809 07:52:24.770671 4730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:24.777077 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 4730
I20260809 07:52:24.777227 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:52:24.777546 18717 external_mini_cluster.cc:1428] Setting key 5f8345ca9f0d5087b601dd2ff939f43c
Aug 09 07:52:24 dist-test-slave-10pc krb5kdc[4614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:24.785756 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:24.772552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:48929 (local address 127.18.71.126:37953)
0809 07:52:24.772704 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:24.772708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:24.773658 (+ 950us) server_negotiation.cc:407] Connection header received
0809 07:52:24.775081 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:24.775084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:24.775143 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:24.775260 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:24.776511 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:24.777240 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:24.778746 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:24.779053 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:24.779177 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:24.781976 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:24.781979 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:24.781982 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:24.782014 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:24.783502 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:24.784053 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:24.784057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:24.784059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:24.784148 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:24.784559 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:24.784562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:24.784564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:24.784762 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:24.784875 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:24.785432 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 07:52:24.785573 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:52:24.786664 4872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:24.773048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37953 (local address 127.18.71.65:48929)
0809 07:52:24.773497 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:24.773541 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:24.774865 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:24.775454 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:24.775462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:24.775827 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:24.776331 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:24.776346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:24.777403 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:24.777406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:24.778633 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:24.778644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:24.779239 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:24.779243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:24.779392 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:24.780037 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:24.780052 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:24.781804 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:24.783630 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:24.783639 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:24.783655 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:24.783933 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:24.784259 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:24.784263 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:24.784264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:24.784413 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:24.784875 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:24.784883 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:24.785176 (+ 293us) client_negotiation.cc:769] Sending connection context
0809 07:52:24.785375 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 07:52:24.785738 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:52:24.787982 4870 heartbeater.cc:347] Connected to a master server at 127.18.71.126:37953
I20260809 07:52:24.788280 4870 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:24.788870 4870 heartbeater.cc:511] Master 127.18.71.126:37953 requested a full tablet report, sending...
I20260809 07:52:24.790433 4664 ts_manager.cc:194] Registered new tserver with Master: a056c81fc8994d61b6420a4588fc7f31 (127.18.71.65:32819)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
I20260809 07:52:24.791651 4664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:48929
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:52:24.808899 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:37953
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:24.923870 4875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:24.924140 4875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:24.924214 4875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:24.928287 4875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:24.928344 4875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:24.928432 4875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:52:24.933749 4875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:37953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:24.934793 4875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:24.935827 4875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:24.945920 4880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:24.946029 4883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:24.945919 4881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:24.946180 4875 server_base.cc:1034] running on GCE node
I20260809 07:52:24.946700 4875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:24.947371 4875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:24.948545 4875 hybrid_clock.cc:648] HybridClock initialized: now 1786261944948515 us; error 34 us; skew 500 ppm
Aug 09 07:52:24 dist-test-slave-10pc krb5kdc[4614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:24.951400 4875 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:52:24.952333 4875 webserver.cc:460] Webserver started at http://127.18.71.66:40891/ using document root <none> and password file <none>
I20260809 07:52:24.952999 4875 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:24.953094 4875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:24.953353 4875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:24.955515 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "84f29334007249b7bdb4e2693c250da1"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "6def2974d25e633e8d73aac56bab5b7a"
server_key_iv: "eb1d0e9057d7b085054d017c95a696d0"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.956260 4875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "84f29334007249b7bdb4e2693c250da1"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "6def2974d25e633e8d73aac56bab5b7a"
server_key_iv: "eb1d0e9057d7b085054d017c95a696d0"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.959949 4875 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:24.962404 4891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.963804 4875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:24.963924 4875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "84f29334007249b7bdb4e2693c250da1"
format_stamp: "Formatted at 2026-08-09 07:52:24 on dist-test-slave-10pc"
server_key: "6def2974d25e633e8d73aac56bab5b7a"
server_key_iv: "eb1d0e9057d7b085054d017c95a696d0"
server_key_version: "encryptionkey@0"
I20260809 07:52:24.964102 4875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:24.982435 4875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:24.986055 4875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:24.986280 4875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:24.986937 4875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:24.987860 4875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:24.987916 4875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.987957 4875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:24.987973 4875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:24.997609 4875 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:41279
I20260809 07:52:24.997642 5011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:41279 every 8 connection(s)
I20260809 07:52:24.998524 4875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:25.006613 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 4875
I20260809 07:52:25.006825 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:52:25.007205 18717 external_mini_cluster.cc:1428] Setting key 47c5035ef8744914a75980ef41817150
Aug 09 07:52:25 dist-test-slave-10pc krb5kdc[4614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261944, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:25.014762 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.000520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:57103 (local address 127.18.71.126:37953)
0809 07:52:25.000689 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.000694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.001574 (+ 880us) server_negotiation.cc:407] Connection header received
0809 07:52:25.003050 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.003052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.003098 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.003178 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:25.004556 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.005090 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.006634 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.006999 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.007083 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.010527 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:25.010531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:25.010533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:25.010556 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:25.012401 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:25.012971 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:25.012977 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:25.012982 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:25.013079 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:25.013426 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:25.013432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:25.013436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:25.013720 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:25.013826 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.014354 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.014517 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 07:52:25.015527 5014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.000907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37953 (local address 127.18.71.66:57103)
0809 07:52:25.001391 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.001455 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.002834 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.003370 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.003377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.003784 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:25.004381 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.004396 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.005215 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.005218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.006502 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.006515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.007174 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.007180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.007412 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.008211 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:25.008231 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:25.010354 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:25.012567 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:25.012572 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:25.012585 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:25.012849 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:25.013178 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:25.013181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:25.013182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:25.013322 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:25.013832 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:25.013837 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:25.014081 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.014235 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.014501 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":117,"threads_started":1}
I20260809 07:52:25.016875 5012 heartbeater.cc:347] Connected to a master server at 127.18.71.126:37953
I20260809 07:52:25.017191 5012 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:25.017704 5012 heartbeater.cc:511] Master 127.18.71.126:37953 requested a full tablet report, sending...
I20260809 07:52:25.018785 4664 ts_manager.cc:194] Registered new tserver with Master: 84f29334007249b7bdb4e2693c250da1 (127.18.71.66:41279)
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:52:25.019426 4664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:57103
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:52:25.039672 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:37953
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:37941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:25.159044 5018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:25.159366 5018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:25.159443 5018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:25.164232 5018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:25.164305 5018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:25.164371 5018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:52:25.171132 5018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:37941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:37953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:25.172245 5018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:25.173413 5018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:25.183727 5025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.183734 5027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.183701 5024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:25.184333 5018 server_base.cc:1034] running on GCE node
I20260809 07:52:25.184811 5018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:25.185520 5018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:25.186723 5018 hybrid_clock.cc:648] HybridClock initialized: now 1786261945186708 us; error 26 us; skew 500 ppm
Aug 09 07:52:25 dist-test-slave-10pc krb5kdc[4614](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:25.189781 5018 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:52:25.190775 5018 webserver.cc:460] Webserver started at http://127.18.71.67:33361/ using document root <none> and password file <none>
I20260809 07:52:25.191390 5018 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:25.191500 5018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:25.191763 5018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:25.194020 5018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "a981d898c30b48cc88a7d493745cf0ff"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "a4eccd05740c554d4d8602482a995a8c"
server_key_iv: "d9b482337ac3d35a8c4ddc6807a097b1"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.194667 5018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "a981d898c30b48cc88a7d493745cf0ff"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "a4eccd05740c554d4d8602482a995a8c"
server_key_iv: "d9b482337ac3d35a8c4ddc6807a097b1"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.198787 5018 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:25.201332 5034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.202536 5018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:25.202711 5018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "a981d898c30b48cc88a7d493745cf0ff"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "a4eccd05740c554d4d8602482a995a8c"
server_key_iv: "d9b482337ac3d35a8c4ddc6807a097b1"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.202828 5018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:25.223858 5018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:25.227281 5018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:25.227502 5018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:25.228109 5018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:25.229063 5018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:25.229111 5018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.229184 5018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:25.229219 5018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.238101 5018 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:40289
I20260809 07:52:25.238168 5151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:40289 every 8 connection(s)
I20260809 07:52:25.239024 5018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:25.247448 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 5018
I20260809 07:52:25.247629 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:52:25.247980 18717 external_mini_cluster.cc:1428] Setting key 8ec6e72f5e267f6767ac286200b370a6
Aug 09 07:52:25 dist-test-slave-10pc krb5kdc[4614](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261945, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:25.253473 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.240880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:40753 (local address 127.18.71.126:37953)
0809 07:52:25.241032 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.241037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.241952 (+ 915us) server_negotiation.cc:407] Connection header received
0809 07:52:25.243375 (+ 1423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.243379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.243439 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.243562 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:25.244822 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.245371 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.246541 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.246769 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.246885 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.249793 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:25.249804 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:25.249806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:25.249830 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:25.251142 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:25.251666 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:25.251670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:25.251671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:25.251755 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:25.252121 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:25.252126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:25.252127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:25.252313 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:25.252428 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.253153 (+ 725us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.253291 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:52:25.254324 5154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.241313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37953 (local address 127.18.71.67:40753)
0809 07:52:25.241802 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.241846 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.243156 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.243746 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.243753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.244114 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:25.244676 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.244692 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.245514 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.245522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.246409 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.246418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.246925 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.246930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.247105 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.247740 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:25.247760 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:25.249632 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:25.251271 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:25.251276 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:25.251291 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:25.251558 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:25.251858 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:25.251861 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:25.251863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:25.252014 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:25.252428 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:25.252436 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:25.252838 (+ 402us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.253085 (+ 247us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.253434 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":112,"threads_started":1}
I20260809 07:52:25.255431 5152 heartbeater.cc:347] Connected to a master server at 127.18.71.126:37953
I20260809 07:52:25.255678 5152 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:25.256181 5152 heartbeater.cc:511] Master 127.18.71.126:37953 requested a full tablet report, sending...
I20260809 07:52:25.257367 4664 ts_manager.cc:194] Registered new tserver with Master: a981d898c30b48cc88a7d493745cf0ff (127.18.71.67:40289)
I20260809 07:52:25.257946 4664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:40753
I20260809 07:52:25.261970 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:52:25.271731 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.264278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37390 (local address 127.18.71.126:37953)
0809 07:52:25.264425 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.264429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.264661 (+ 232us) server_negotiation.cc:407] Connection header received
0809 07:52:25.264810 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.264813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.264867 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.264954 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:25.265631 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.266184 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.266989 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.267223 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.267361 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.268785 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:25.268788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:25.268790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:25.268811 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:25.270319 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:25.270693 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:25.270695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:25.270696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:25.270767 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:25.270998 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:25.271001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:25.271003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:25.271206 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:25.271306 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.271499 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.271584 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:52:25.274528 4664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:52:25.277917 4664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:52:25.292147 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.286365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:40289 (local address 127.0.0.1:45880)
0809 07:52:25.286889 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.286939 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.287096 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.288225 (+ 1129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.288231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.288254 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:25.288393 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.288400 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.289471 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.289477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.290600 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.290612 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.291416 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.291420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.291700 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.291780 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.291888 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.292003 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":82,"thread_start_us":74,"threads_started":1}
I20260809 07:52:25.292650 5166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.286528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45880 (local address 127.18.71.67:40289)
0809 07:52:25.287680 (+ 1152us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.287696 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.287722 (+ 26us) server_negotiation.cc:407] Connection header received
0809 07:52:25.287805 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.287812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.288061 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.288233 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:25.288546 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.289335 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.290732 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.291284 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.291404 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.291561 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.292345 (+ 784us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.292444 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":210,"threads_started":1}
I20260809 07:52:25.293036 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.287142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:41279 (local address 127.0.0.1:60444)
0809 07:52:25.287687 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.287703 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.287786 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.288402 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.288405 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.288423 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:25.288546 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.288553 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.290125 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.290129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.291538 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.291552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.292684 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.292690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.292802 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.292828 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.292883 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.292944 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":2,"thread_start_us":135,"threads_started":1}
I20260809 07:52:25.293301 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.286538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:32819 (local address 127.0.0.1:36950)
0809 07:52:25.287483 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.287498 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.287594 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.288523 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.288526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.288546 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:25.288677 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.288685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.290290 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.290293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.291778 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.291786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.292879 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.292883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.293008 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.293034 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.293092 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.293164 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":170,"thread_start_us":102,"threads_started":1}
I20260809 07:52:25.293835 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.287292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.18.71.66:41279)
0809 07:52:25.287880 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.287893 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.287917 (+ 24us) server_negotiation.cc:407] Connection header received
0809 07:52:25.288007 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.288013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.288213 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.288430 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:25.288631 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.289949 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.291691 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.292540 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.293370 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.293526 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.293577 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.293667 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":107,"threads_started":1}
I20260809 07:52:25.294334 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.287183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.18.71.65:32819)
0809 07:52:25.288030 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.288053 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.288074 (+ 21us) server_negotiation.cc:407] Connection header received
0809 07:52:25.288143 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.288148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.288319 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.288882 (+ 563us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:25.289023 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.290165 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.291895 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.292724 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.293863 (+ 1139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.294040 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.294099 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.294191 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":95,"threads_started":1}
I20260809 07:52:25.296190 5085 tablet_service.cc:1467] Processing CreateTablet for tablet e94361637b074146b83032b2968de769 (DEFAULT_TABLE table=test-table [id=27994065e6f347ef8372d461bc87e544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:25.296650 4799 tablet_service.cc:1467] Processing CreateTablet for tablet e94361637b074146b83032b2968de769 (DEFAULT_TABLE table=test-table [id=27994065e6f347ef8372d461bc87e544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:25.296898 4943 tablet_service.cc:1467] Processing CreateTablet for tablet e94361637b074146b83032b2968de769 (DEFAULT_TABLE table=test-table [id=27994065e6f347ef8372d461bc87e544]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:25.297490 5085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e94361637b074146b83032b2968de769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:25.297695 4799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e94361637b074146b83032b2968de769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:25.297917 4943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e94361637b074146b83032b2968de769. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:25.303658 5170 tablet_bootstrap.cc:492] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31: Bootstrap starting.
I20260809 07:52:25.305173 5171 tablet_bootstrap.cc:492] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Bootstrap starting.
I20260809 07:52:25.305583 5172 tablet_bootstrap.cc:492] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Bootstrap starting.
I20260809 07:52:25.306555 5170 tablet_bootstrap.cc:654] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:25.307477 5170 log.cc:824] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:25.308337 5172 tablet_bootstrap.cc:654] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:25.308611 5171 tablet_bootstrap.cc:654] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:25.309113 5172 log.cc:824] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:25.309980 5170 tablet_bootstrap.cc:492] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31: No bootstrap required, opened a new log
I20260809 07:52:25.309994 5171 log.cc:824] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:25.310200 5170 ts_tablet_manager.cc:1397] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 07:52:25.310979 5172 tablet_bootstrap.cc:492] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: No bootstrap required, opened a new log
I20260809 07:52:25.311178 5172 ts_tablet_manager.cc:1397] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:52:25.312032 5171 tablet_bootstrap.cc:492] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: No bootstrap required, opened a new log
I20260809 07:52:25.312273 5171 ts_tablet_manager.cc:1397] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260809 07:52:25.314142 5172 raft_consensus.cc:348] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.314533 5172 raft_consensus.cc:374] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:25.314656 5172 raft_consensus.cc:729] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f29334007249b7bdb4e2693c250da1, State: Initialized, Role: FOLLOWER
I20260809 07:52:25.315209 5172 consensus_queue.cc:260] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.315150 5170 raft_consensus.cc:348] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.315449 5170 raft_consensus.cc:374] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:25.315573 5170 raft_consensus.cc:729] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a056c81fc8994d61b6420a4588fc7f31, State: Initialized, Role: FOLLOWER
I20260809 07:52:25.315887 5012 heartbeater.cc:503] Master 127.18.71.126:37953 was elected leader, sending a full tablet report...
I20260809 07:52:25.316200 5172 ts_tablet_manager.cc:1428] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 07:52:25.316416 5171 raft_consensus.cc:348] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.316766 5171 raft_consensus.cc:374] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:25.316847 5171 raft_consensus.cc:729] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a981d898c30b48cc88a7d493745cf0ff, State: Initialized, Role: FOLLOWER
I20260809 07:52:25.316898 5170 consensus_queue.cc:260] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.317265 5171 consensus_queue.cc:260] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.317668 4870 heartbeater.cc:503] Master 127.18.71.126:37953 was elected leader, sending a full tablet report...
I20260809 07:52:25.317880 5170 ts_tablet_manager.cc:1428] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260809 07:52:25.318262 5171 ts_tablet_manager.cc:1428] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 07:52:25.318606 5152 heartbeater.cc:503] Master 127.18.71.126:37953 was elected leader, sending a full tablet report...
I20260809 07:52:25.399674 5176 raft_consensus.cc:482] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:25.399870 5176 raft_consensus.cc:504] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.401084 5176 leader_election.cc:288] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a981d898c30b48cc88a7d493745cf0ff (127.18.71.67:40289), a056c81fc8994d61b6420a4588fc7f31 (127.18.71.65:32819)
I20260809 07:52:25.406226 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.401636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:49873 (local address 127.18.71.65:32819)
0809 07:52:25.401775 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.401780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.401899 (+ 119us) server_negotiation.cc:407] Connection header received
0809 07:52:25.401988 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.401991 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.402045 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.402124 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:25.402447 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.403270 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.405207 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.405642 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.405736 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.405775 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.406047 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.406129 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 07:52:25.406247 5179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.401415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:32819 (local address 127.18.71.66:49873)
0809 07:52:25.401815 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.401826 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.401908 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.402148 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.402151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.402189 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:25.402320 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.402327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.403380 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.403383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.405035 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.405048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.405763 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.405769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.405894 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.405927 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.406002 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.406072 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":197,"spinlock_wait_cycles":2688,"thread_start_us":171,"threads_started":1}
I20260809 07:52:25.406486 5166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.401690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:33487 (local address 127.18.71.67:40289)
0809 07:52:25.401837 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.401842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.401856 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:52:25.401923 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.401926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.401978 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.402068 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:25.402447 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.403243 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.405204 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.405712 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.405830 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.405887 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.406303 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.406389 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:52:25.406687 5014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.401280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:40289 (local address 127.18.71.66:33487)
0809 07:52:25.401395 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.401414 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.401520 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.402088 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.402091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.402140 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:25.402319 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.402327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.403380 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.403383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.405043 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.405049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.405941 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.405944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.406058 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.406076 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.406148 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.406205 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":181,"spinlock_wait_cycles":14208}
I20260809 07:52:25.407368 4820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e94361637b074146b83032b2968de769" candidate_uuid: "84f29334007249b7bdb4e2693c250da1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a056c81fc8994d61b6420a4588fc7f31" is_pre_election: true
I20260809 07:52:25.407368 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e94361637b074146b83032b2968de769" candidate_uuid: "84f29334007249b7bdb4e2693c250da1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981d898c30b48cc88a7d493745cf0ff" is_pre_election: true
I20260809 07:52:25.407709 5106 raft_consensus.cc:2454] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84f29334007249b7bdb4e2693c250da1 in term 0.
I20260809 07:52:25.407709 4820 raft_consensus.cc:2454] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84f29334007249b7bdb4e2693c250da1 in term 0.
I20260809 07:52:25.408188 4892 leader_election.cc:302] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84f29334007249b7bdb4e2693c250da1, a981d898c30b48cc88a7d493745cf0ff; no voters:
I20260809 07:52:25.408430 5176 raft_consensus.cc:2781] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:52:25.408551 5176 raft_consensus.cc:482] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:25.408615 5176 raft_consensus.cc:3037] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:25.409663 5176 raft_consensus.cc:504] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.410012 5176 leader_election.cc:288] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [CANDIDATE]: Term 1 election: Requested vote from peers a981d898c30b48cc88a7d493745cf0ff (127.18.71.67:40289), a056c81fc8994d61b6420a4588fc7f31 (127.18.71.65:32819)
I20260809 07:52:25.410384 4820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e94361637b074146b83032b2968de769" candidate_uuid: "84f29334007249b7bdb4e2693c250da1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a056c81fc8994d61b6420a4588fc7f31"
I20260809 07:52:25.410384 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e94361637b074146b83032b2968de769" candidate_uuid: "84f29334007249b7bdb4e2693c250da1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a981d898c30b48cc88a7d493745cf0ff"
I20260809 07:52:25.410514 4820 raft_consensus.cc:3037] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:25.410538 5106 raft_consensus.cc:3037] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:25.411830 5106 raft_consensus.cc:2454] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84f29334007249b7bdb4e2693c250da1 in term 1.
I20260809 07:52:25.411830 4820 raft_consensus.cc:2454] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84f29334007249b7bdb4e2693c250da1 in term 1.
I20260809 07:52:25.412140 4892 leader_election.cc:302] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84f29334007249b7bdb4e2693c250da1, a981d898c30b48cc88a7d493745cf0ff; no voters:
I20260809 07:52:25.412369 5176 raft_consensus.cc:2781] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:25.412624 5176 raft_consensus.cc:686] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 1 LEADER]: Becoming Leader. State: Replica: 84f29334007249b7bdb4e2693c250da1, State: Running, Role: LEADER
I20260809 07:52:25.412971 5176 consensus_queue.cc:237] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } }
I20260809 07:52:25.416198 4664 catalog_manager.cc:5495] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84f29334007249b7bdb4e2693c250da1 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "84f29334007249b7bdb4e2693c250da1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84f29334007249b7bdb4e2693c250da1" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 41279 } health_report { overall_health: HEALTHY } } }
I20260809 07:52:25.443368 5169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.439400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.18.71.66:41279)
0809 07:52:25.439539 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.439542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.439556 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:52:25.439648 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.439650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.439705 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.439788 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:52:25.440063 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.440656 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.441894 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.442109 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.442225 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.442572 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:52:25.442972 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:52:25.443104 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.443166 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.443221 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 07:52:25.450093 4820 raft_consensus.cc:1261] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 1 FOLLOWER]: Refusing update from remote peer 84f29334007249b7bdb4e2693c250da1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:25.450096 5106 raft_consensus.cc:1261] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 1 FOLLOWER]: Refusing update from remote peer 84f29334007249b7bdb4e2693c250da1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:25.450881 5180 consensus_queue.cc:1035] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a981d898c30b48cc88a7d493745cf0ff" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 40289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:25.451087 5176 consensus_queue.cc:1035] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a056c81fc8994d61b6420a4588fc7f31" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 32819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:25.459923 5184 mvcc.cc:205] Tried to move back new op lower bound from 7316528928555307008 to 7316528928413995008. Current Snapshot: MvccSnapshot[applied={T|T < 7316528928555307008}]
I20260809 07:52:25.468727 4662 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:25.468926 4662 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:25.471441 4662 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de4ea8a59b7049c68558a356bda8f6d5: Sending DeleteTablet for 3 replicas of tablet e94361637b074146b83032b2968de769
I20260809 07:52:25.472247 5085 tablet_service.cc:1514] Processing DeleteTablet for tablet e94361637b074146b83032b2968de769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:25 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:45880
I20260809 07:52:25.472240 4799 tablet_service.cc:1514] Processing DeleteTablet for tablet e94361637b074146b83032b2968de769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:25 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:36950
I20260809 07:52:25.472807 5195 tablet_replica.cc:330] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31: stopping tablet replica
I20260809 07:52:25.472810 4943 tablet_service.cc:1514] Processing DeleteTablet for tablet e94361637b074146b83032b2968de769 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:25 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:60444
I20260809 07:52:25.473145 5195 raft_consensus.cc:2229] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:25.473269 5197 tablet_replica.cc:330] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: stopping tablet replica
I20260809 07:52:25.473558 5197 raft_consensus.cc:2229] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:52:25.473560 5195 raft_consensus.cc:2258] T e94361637b074146b83032b2968de769 P a056c81fc8994d61b6420a4588fc7f31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:25.474037 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 4730
I20260809 07:52:25.474632 5193 tablet_replica.cc:330] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: stopping tablet replica
I20260809 07:52:25.474639 5197 raft_consensus.cc:2258] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:25.474977 5193 raft_consensus.cc:2229] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:25.475311 5193 raft_consensus.cc:2258] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:25.476593 5197 ts_tablet_manager.cc:1905] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:25.476593 5193 ts_tablet_manager.cc:1905] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:25.479380 5197 ts_tablet_manager.cc:1918] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:25.479368 5193 ts_tablet_manager.cc:1918] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:25.479491 5193 log.cc:1196] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/e94361637b074146b83032b2968de769
I20260809 07:52:25.479568 5197 log.cc:1196] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/e94361637b074146b83032b2968de769
I20260809 07:52:25.479951 5197 ts_tablet_manager.cc:1939] T e94361637b074146b83032b2968de769 P 84f29334007249b7bdb4e2693c250da1: Deleting consensus metadata
I20260809 07:52:25.479951 5193 ts_tablet_manager.cc:1939] T e94361637b074146b83032b2968de769 P a981d898c30b48cc88a7d493745cf0ff: Deleting consensus metadata
I20260809 07:52:25.481189 4647 catalog_manager.cc:4841] TS a981d898c30b48cc88a7d493745cf0ff (127.18.71.67:40289): tablet e94361637b074146b83032b2968de769 (table test-table [id=27994065e6f347ef8372d461bc87e544]) successfully deleted
I20260809 07:52:25.484678 4648 catalog_manager.cc:4841] TS 84f29334007249b7bdb4e2693c250da1 (127.18.71.66:41279): tablet e94361637b074146b83032b2968de769 (table test-table [id=27994065e6f347ef8372d461bc87e544]) successfully deleted
W20260809 07:52:25.487248 4649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:32819 (error 108)
I20260809 07:52:25.487475 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 4875
I20260809 07:52:25.488881 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.488559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:32819 (local address 127.0.0.1:36954)
0809 07:52:25.488693 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.488776 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:32819: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 07:52:25.489049 4649 catalog_manager.cc:4584] TS a056c81fc8994d61b6420a4588fc7f31 (127.18.71.65:32819): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e94361637b074146b83032b2968de769: Network error: Client connection negotiation failed: client connection to 127.18.71.65:32819: connect: Connection refused (error 111)
I20260809 07:52:25.495079 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 5018
I20260809 07:52:25.502048 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 4630
2026-08-09T07:52:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2466 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T07:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:25Z Disabled control of system clock
I20260809 07:52:25.535627 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33209
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 07:52:25.650141 5203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:25.650398 5203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:25.650432 5203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:25.654336 5203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:25.654392 5203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:25.654407 5203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:25.654423 5203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:25.654438 5203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:25.659210 5203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:25.660246 5203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:25.661615 5203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:25.669943 5211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.669925 5208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.669921 5209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:25.670100 5203 server_base.cc:1034] running on GCE node
I20260809 07:52:25.670899 5203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:25.671990 5203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:25.673168 5203 hybrid_clock.cc:648] HybridClock initialized: now 1786261945673149 us; error 33 us; skew 500 ppm
I20260809 07:52:25.674935 5203 webserver.cc:460] Webserver started at http://127.18.71.126:44247/ using document root <none> and password file <none>
I20260809 07:52:25.675473 5203 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:25.675531 5203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:25.675697 5203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:25.677770 5203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "2a28d3abaf1c4481b10c3042800e24ec"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "383dd048b50cc05b83381f85c394de0d"
server_key_iv: "8613d66aa25da47c0e2bf052c88f2398"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.678299 5203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "2a28d3abaf1c4481b10c3042800e24ec"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "383dd048b50cc05b83381f85c394de0d"
server_key_iv: "8613d66aa25da47c0e2bf052c88f2398"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.681880 5203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:25.684230 5217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.685509 5203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:25.685608 5203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "2a28d3abaf1c4481b10c3042800e24ec"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "383dd048b50cc05b83381f85c394de0d"
server_key_iv: "8613d66aa25da47c0e2bf052c88f2398"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.685691 5203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:25.698727 5203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:25.699301 5203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:25.699437 5203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:25.706213 5203 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33209
I20260809 07:52:25.706228 5272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33209 every 8 connection(s)
I20260809 07:52:25.707129 5203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:25.710291 5273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:25.711685 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 5203
I20260809 07:52:25.711828 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:25.712136 18717 external_mini_cluster.cc:1428] Setting key 1217fa629f26ea71a91235afe9bef427
I20260809 07:52:25.717172 5273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec: Bootstrap starting.
I20260809 07:52:25.719774 5273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:25.720585 5273 log.cc:824] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:25.722735 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.713601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.18.71.126:33209)
0809 07:52:25.714302 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.714323 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.714369 (+ 46us) server_negotiation.cc:407] Connection header received
0809 07:52:25.715263 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.715287 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.715782 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.716171 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:25.716644 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.718092 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.718971 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.719655 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.719834 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.720712 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:25.720720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:25.720736 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:25.721225 (+ 489us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:25.721278 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:25.721301 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:25.721425 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.721577 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.721841 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":113,"threads_started":1}
I20260809 07:52:25.722851 5273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec: No bootstrap required, opened a new log
I20260809 07:52:25.726650 5273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a28d3abaf1c4481b10c3042800e24ec" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33209 } }
I20260809 07:52:25.726861 5273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:25.726892 5273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a28d3abaf1c4481b10c3042800e24ec, State: Initialized, Role: FOLLOWER
I20260809 07:52:25.727284 5273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a28d3abaf1c4481b10c3042800e24ec" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33209 } }
I20260809 07:52:25.727403 5273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:25.727448 5273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:25.727517 5273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:25.728328 5273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a28d3abaf1c4481b10c3042800e24ec" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33209 } }
I20260809 07:52:25.728670 5273 leader_election.cc:302] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a28d3abaf1c4481b10c3042800e24ec; no voters:
I20260809 07:52:25.728914 5273 leader_election.cc:288] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:25.729069 5278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:25.729315 5278 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [term 1 LEADER]: Becoming Leader. State: Replica: 2a28d3abaf1c4481b10c3042800e24ec, State: Running, Role: LEADER
I20260809 07:52:25.729673 5278 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a28d3abaf1c4481b10c3042800e24ec" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33209 } }
I20260809 07:52:25.730145 5273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:25.731226 5280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a28d3abaf1c4481b10c3042800e24ec. Latest consensus state: current_term: 1 leader_uuid: "2a28d3abaf1c4481b10c3042800e24ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a28d3abaf1c4481b10c3042800e24ec" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33209 } } }
I20260809 07:52:25.731541 5279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a28d3abaf1c4481b10c3042800e24ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a28d3abaf1c4481b10c3042800e24ec" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33209 } } }
I20260809 07:52:25.731658 5280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:25.731725 5279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:25.735141 5294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:25.735206 5294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:25.735285 5289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:25.736070 5289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:25.741021 5289 catalog_manager.cc:1347] Generated new cluster ID: 15c2eabd9d2c40b49c1561eb2d6ee3ff
I20260809 07:52:25.741083 5289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:25.746810 5289 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:25.747668 5289 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:25.753834 5289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec: Generated new TSK 0
I20260809 07:52:25.754446 5289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:25.759335 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33209
--builtin_ntp_servers=127.18.71.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 07:52:25.873149 5298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:25.873557 5298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:25.873637 5298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:25.879976 5298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:25.880057 5298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:25.880146 5298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:52:25.885540 5298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:25.886643 5298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:25.888022 5298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:25.897709 5303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.897725 5306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:25.897750 5304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:25.897926 5298 server_base.cc:1034] running on GCE node
I20260809 07:52:25.898301 5298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:25.898958 5298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:25.900152 5298 hybrid_clock.cc:648] HybridClock initialized: now 1786261945900137 us; error 27 us; skew 500 ppm
I20260809 07:52:25.902163 5298 webserver.cc:460] Webserver started at http://127.18.71.65:36411/ using document root <none> and password file <none>
I20260809 07:52:25.902740 5298 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:25.902832 5298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:25.903070 5298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:25.905205 5298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "2dfdd9bbb9da4403b515d5ee8e398e53"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "de65053f30daa681665a35e8b62999b8"
server_key_iv: "badc07d416add5569c32cac3b437616d"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.905771 5298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "2dfdd9bbb9da4403b515d5ee8e398e53"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "de65053f30daa681665a35e8b62999b8"
server_key_iv: "badc07d416add5569c32cac3b437616d"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.909354 5298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:52:25.911788 5313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.913066 5298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:25.913220 5298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "2dfdd9bbb9da4403b515d5ee8e398e53"
format_stamp: "Formatted at 2026-08-09 07:52:25 on dist-test-slave-10pc"
server_key: "de65053f30daa681665a35e8b62999b8"
server_key_iv: "badc07d416add5569c32cac3b437616d"
server_key_version: "encryptionkey@0"
I20260809 07:52:25.913334 5298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:25.927675 5298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:25.928294 5298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:25.928541 5298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:25.929104 5298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:25.930110 5298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:25.930158 5298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.930223 5298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:25.930255 5298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:25.938735 5298 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:37721
I20260809 07:52:25.938832 5434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:37721 every 8 connection(s)
I20260809 07:52:25.939663 5298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:25.946748 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 5298
I20260809 07:52:25.946868 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:52:25.947247 18717 external_mini_cluster.cc:1428] Setting key f44f2f151af08cab4c701fc29c03b392
I20260809 07:52:25.949438 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.941488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:39261 (local address 127.18.71.126:33209)
0809 07:52:25.941627 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:25.941631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:25.942630 (+ 999us) server_negotiation.cc:407] Connection header received
0809 07:52:25.944025 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:25.944028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:25.944084 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:25.944175 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:25.945256 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.945815 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.947062 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.947459 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.947572 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.948879 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:25.948883 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:25.948885 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:25.948900 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:25.948925 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:25.948934 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:25.949018 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:25.949172 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 07:52:25.949246 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:52:25.949554 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33209
--builtin_ntp_servers=127.18.71.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 07:52:25.950259 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:25.941843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33209 (local address 127.18.71.65:39261)
0809 07:52:25.942487 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:25.942526 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:25.943785 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:25.944381 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:25.944388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:25.944538 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:25.945105 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.945121 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.945950 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.945953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.946916 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:25.946947 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:25.947628 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:25.947651 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:25.947883 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:25.948550 (+ 667us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:25.948552 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:25.948610 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:25.948693 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:25.949027 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:25.949035 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:25.949040 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:52:25.949150 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 07:52:25.949378 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":121,"threads_started":1}
I20260809 07:52:25.951562 5435 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33209
I20260809 07:52:25.951810 5435 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:25.952373 5435 heartbeater.cc:511] Master 127.18.71.126:33209 requested a full tablet report, sending...
I20260809 07:52:25.953877 5236 ts_manager.cc:194] Registered new tserver with Master: 2dfdd9bbb9da4403b515d5ee8e398e53 (127.18.71.65:37721)
I20260809 07:52:25.955051 5236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.65:39261
W20260809 07:52:26.067962 5438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:26.068264 5438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:26.068336 5438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:26.072352 5438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:26.072441 5438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:26.072561 5438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:52:26.077450 5438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:26.078584 5438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:26.080010 5438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:26.090384 5449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:26.090422 5446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:26.090634 5438 server_base.cc:1034] running on GCE node
W20260809 07:52:26.090412 5445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:26.091187 5438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:26.091837 5438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:26.093001 5438 hybrid_clock.cc:648] HybridClock initialized: now 1786261946092985 us; error 30 us; skew 500 ppm
I20260809 07:52:26.094947 5438 webserver.cc:460] Webserver started at http://127.18.71.66:44855/ using document root <none> and password file <none>
I20260809 07:52:26.095516 5438 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:26.095579 5438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:26.095748 5438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:26.097949 5438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "a2c33c6fa8704eb6804ec7b862967af0"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-10pc"
server_key: "14651676737f9c0140b084713a7b9f4f"
server_key_iv: "16cfa3044a0046de526529bc2ac67ea7"
server_key_version: "encryptionkey@0"
I20260809 07:52:26.098513 5438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "a2c33c6fa8704eb6804ec7b862967af0"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-10pc"
server_key: "14651676737f9c0140b084713a7b9f4f"
server_key_iv: "16cfa3044a0046de526529bc2ac67ea7"
server_key_version: "encryptionkey@0"
I20260809 07:52:26.102244 5438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:26.104758 5455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.105999 5438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:26.106096 5438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "a2c33c6fa8704eb6804ec7b862967af0"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-10pc"
server_key: "14651676737f9c0140b084713a7b9f4f"
server_key_iv: "16cfa3044a0046de526529bc2ac67ea7"
server_key_version: "encryptionkey@0"
I20260809 07:52:26.106191 5438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:26.129644 5438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:26.130344 5438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:26.130533 5438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:26.131213 5438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:26.132289 5438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:26.132341 5438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.132383 5438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:26.132398 5438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.142076 5438 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:39385
I20260809 07:52:26.142113 5576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:39385 every 8 connection(s)
I20260809 07:52:26.143064 5438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:26.148070 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 5438
I20260809 07:52:26.148217 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:52:26.148757 18717 external_mini_cluster.cc:1428] Setting key 3e4f3c5c5955b62b6a9aae5b1051b565
I20260809 07:52:26.151150 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33209
--builtin_ntp_servers=127.18.71.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 07:52:26.154053 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.145054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:34989 (local address 127.18.71.126:33209)
0809 07:52:26.145198 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.145202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.146328 (+ 1126us) server_negotiation.cc:407] Connection header received
0809 07:52:26.147818 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.147821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.147881 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.147989 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:26.149155 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.149887 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.151791 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.152010 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.152115 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.153458 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:26.153464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:26.153469 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:26.153486 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:26.153514 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:26.153527 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:26.153610 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.153776 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.153854 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 07:52:26.154834 5579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.145508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33209 (local address 127.18.71.66:34989)
0809 07:52:26.146179 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.146219 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.147576 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.148196 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.148206 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.148339 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:26.149004 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.149021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.150160 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.150167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.151655 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.151671 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.152182 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.152189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.152424 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.153136 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:26.153140 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:26.153204 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:26.153289 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:26.153626 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:26.153634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:26.153638 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.153738 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.153970 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":116,"threads_started":1}
I20260809 07:52:26.156037 5577 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33209
I20260809 07:52:26.156318 5577 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:26.157030 5577 heartbeater.cc:511] Master 127.18.71.126:33209 requested a full tablet report, sending...
I20260809 07:52:26.158281 5236 ts_manager.cc:194] Registered new tserver with Master: a2c33c6fa8704eb6804ec7b862967af0 (127.18.71.66:39385)
I20260809 07:52:26.158870 5236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.66:34989
W20260809 07:52:26.272342 5580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:26.272677 5580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:26.272722 5580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:26.276801 5580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:26.276855 5580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:26.276909 5580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:52:26.281773 5580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:26.282869 5580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:26.284274 5580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:26.294102 5589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:26.294270 5580 server_base.cc:1034] running on GCE node
W20260809 07:52:26.294114 5587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:26.294114 5586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:26.295068 5580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:26.295790 5580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:26.296986 5580 hybrid_clock.cc:648] HybridClock initialized: now 1786261946296970 us; error 29 us; skew 500 ppm
I20260809 07:52:26.299001 5580 webserver.cc:460] Webserver started at http://127.18.71.67:41589/ using document root <none> and password file <none>
I20260809 07:52:26.299620 5580 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:26.299726 5580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:26.299981 5580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:26.302268 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-10pc"
server_key: "cb4988abbc187939e3d86149eb56f392"
server_key_iv: "bc463380705b15f728b5146b31a53c5f"
server_key_version: "encryptionkey@0"
I20260809 07:52:26.302894 5580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-10pc"
server_key: "cb4988abbc187939e3d86149eb56f392"
server_key_iv: "bc463380705b15f728b5146b31a53c5f"
server_key_version: "encryptionkey@0"
I20260809 07:52:26.306835 5580 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 07:52:26.309504 5595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.310870 5580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:26.310992 5580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4"
format_stamp: "Formatted at 2026-08-09 07:52:26 on dist-test-slave-10pc"
server_key: "cb4988abbc187939e3d86149eb56f392"
server_key_iv: "bc463380705b15f728b5146b31a53c5f"
server_key_version: "encryptionkey@0"
I20260809 07:52:26.311103 5580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:26.332219 5580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:26.333053 5580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:26.333218 5580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:26.333773 5580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:26.334714 5580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:26.334762 5580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.334801 5580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:26.334816 5580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:26.343345 5580 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:43955
I20260809 07:52:26.343434 5716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:43955 every 8 connection(s)
I20260809 07:52:26.344319 5580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:26.349331 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 5580
I20260809 07:52:26.349426 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:52:26.349705 18717 external_mini_cluster.cc:1428] Setting key e163a28196325313c9f24b63c17cd9b8
I20260809 07:52:26.354254 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.346222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:53859 (local address 127.18.71.126:33209)
0809 07:52:26.346395 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.346399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.347439 (+ 1040us) server_negotiation.cc:407] Connection header received
0809 07:52:26.348928 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.348932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.349002 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.349108 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:26.350203 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.350985 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.352351 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.352620 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.352706 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.353809 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:26.353812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:26.353814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:26.353825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:26.353843 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:26.353851 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:26.353911 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.354088 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.354146 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 07:52:26.355098 5719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.346598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33209 (local address 127.18.71.67:53859)
0809 07:52:26.347276 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.347318 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.348706 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.349296 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.349303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.349406 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:26.350014 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.350036 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.351142 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.351145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.352249 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.352258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.352785 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.352788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.352939 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.353552 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:26.353553 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:26.353593 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:26.353668 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:26.353929 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:26.353935 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:26.353940 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.354072 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.354290 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":118,"threads_started":1}
I20260809 07:52:26.356211 5717 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33209
I20260809 07:52:26.356503 5717 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:26.357044 5717 heartbeater.cc:511] Master 127.18.71.126:33209 requested a full tablet report, sending...
I20260809 07:52:26.358019 5236 ts_manager.cc:194] Registered new tserver with Master: f6495ee2c3ca44faa6c58cdc8e7629d4 (127.18.71.67:43955)
I20260809 07:52:26.358640 5236 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.67:53859
I20260809 07:52:26.363584 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:52:26.369316 5276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.365626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.18.71.126:33209)
0809 07:52:26.365745 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.365749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.365926 (+ 177us) server_negotiation.cc:407] Connection header received
0809 07:52:26.366022 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.366024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.366059 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.366131 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:26.366366 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.366842 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.367641 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.367836 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.367908 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.368934 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:26.368937 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:26.368939 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:26.368950 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:26.368967 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:26.368974 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:26.369032 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.369174 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.369218 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:52:26.372015 5236 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:52:26.375155 5236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:52:26.390800 5731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.385154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:43955 (local address 127.0.0.1:53900)
0809 07:52:26.385632 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.385646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.385736 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.386398 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.386404 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.386427 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:26.386767 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.386774 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.388125 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.388128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.389279 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.389289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.390303 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.390307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.390391 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.390423 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.390529 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.390624 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":89,"thread_start_us":94,"threads_started":1}
I20260809 07:52:26.390841 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.385365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53900 (local address 127.18.71.67:43955)
0809 07:52:26.385919 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.385933 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.385960 (+ 27us) server_negotiation.cc:407] Connection header received
0809 07:52:26.386061 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.386065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.386226 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.386405 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:26.386877 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.388000 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.389403 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.390148 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.390275 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.390461 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.390559 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.390662 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":185,"threads_started":1}
I20260809 07:52:26.392616 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.384589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39385 (local address 127.0.0.1:43624)
0809 07:52:26.385228 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.385281 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.385407 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.386436 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.386439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.386457 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:26.386616 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.386624 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.388055 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.388061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.389789 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.389801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.392145 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.392150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.392246 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.392272 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.392363 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.392436 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":165,"thread_start_us":96,"threads_started":1}
I20260809 07:52:26.392728 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.385585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.18.71.66:39385)
0809 07:52:26.386047 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.386058 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.386078 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:52:26.386142 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.386146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.386286 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.386595 (+ 309us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:26.386730 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.387909 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.391353 (+ 3444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.392017 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.392103 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.392232 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.392415 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.392556 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":152,"threads_started":1}
I20260809 07:52:26.392768 5729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.384363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37721 (local address 127.0.0.1:54082)
0809 07:52:26.385758 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.385777 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.385893 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.386559 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.386571 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.386585 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:26.386663 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.386669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.387962 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.387965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.389790 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.389801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.392288 (+ 2487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.392291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.392399 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.392424 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.392562 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.392635 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":177,"thread_start_us":128,"threads_started":1}
I20260809 07:52:26.393241 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.384580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54082 (local address 127.18.71.65:37721)
0809 07:52:26.385683 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.385696 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.385871 (+ 175us) server_negotiation.cc:407] Connection header received
0809 07:52:26.386031 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.386037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.386247 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.386405 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:26.386805 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.387825 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.389944 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.390738 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.390844 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.391294 (+ 450us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.392976 (+ 1682us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.393083 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":983,"thread_start_us":185,"threads_started":1}
I20260809 07:52:26.394264 5646 tablet_service.cc:1467] Processing CreateTablet for tablet 19675690a83f4d3dbd63e3d8db097b08 (DEFAULT_TABLE table=test-table [id=c60835a2516e498d85443e4d50a9259e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:26.395085 5503 tablet_service.cc:1467] Processing CreateTablet for tablet 19675690a83f4d3dbd63e3d8db097b08 (DEFAULT_TABLE table=test-table [id=c60835a2516e498d85443e4d50a9259e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:26.395531 5646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19675690a83f4d3dbd63e3d8db097b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:26.395839 5503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19675690a83f4d3dbd63e3d8db097b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:26.396078 5364 tablet_service.cc:1467] Processing CreateTablet for tablet 19675690a83f4d3dbd63e3d8db097b08 (DEFAULT_TABLE table=test-table [id=c60835a2516e498d85443e4d50a9259e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:26.397130 5364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19675690a83f4d3dbd63e3d8db097b08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:26.401937 5734 tablet_bootstrap.cc:492] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Bootstrap starting.
I20260809 07:52:26.403352 5735 tablet_bootstrap.cc:492] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Bootstrap starting.
I20260809 07:52:26.404405 5734 tablet_bootstrap.cc:654] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:26.404968 5736 tablet_bootstrap.cc:492] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53: Bootstrap starting.
I20260809 07:52:26.405658 5735 tablet_bootstrap.cc:654] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:26.405664 5734 log.cc:824] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:26.406558 5735 log.cc:824] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:26.407456 5736 tablet_bootstrap.cc:654] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:26.408349 5735 tablet_bootstrap.cc:492] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: No bootstrap required, opened a new log
I20260809 07:52:26.408396 5736 log.cc:824] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:26.408619 5735 ts_tablet_manager.cc:1397] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 07:52:26.408686 5734 tablet_bootstrap.cc:492] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: No bootstrap required, opened a new log
I20260809 07:52:26.408903 5734 ts_tablet_manager.cc:1397] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260809 07:52:26.410699 5736 tablet_bootstrap.cc:492] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53: No bootstrap required, opened a new log
I20260809 07:52:26.410887 5736 ts_tablet_manager.cc:1397] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:52:26.412402 5735 raft_consensus.cc:348] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.412717 5735 raft_consensus.cc:374] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:26.412775 5735 raft_consensus.cc:729] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6495ee2c3ca44faa6c58cdc8e7629d4, State: Initialized, Role: FOLLOWER
I20260809 07:52:26.413241 5735 consensus_queue.cc:260] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.413859 5717 heartbeater.cc:503] Master 127.18.71.126:33209 was elected leader, sending a full tablet report...
I20260809 07:52:26.414247 5735 ts_tablet_manager.cc:1428] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:52:26.414248 5734 raft_consensus.cc:348] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.414495 5734 raft_consensus.cc:374] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:26.414546 5734 raft_consensus.cc:729] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c33c6fa8704eb6804ec7b862967af0, State: Initialized, Role: FOLLOWER
I20260809 07:52:26.415016 5734 consensus_queue.cc:260] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.415406 5736 raft_consensus.cc:348] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.415664 5736 raft_consensus.cc:374] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:26.415726 5736 raft_consensus.cc:729] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dfdd9bbb9da4403b515d5ee8e398e53, State: Initialized, Role: FOLLOWER
I20260809 07:52:26.415732 5577 heartbeater.cc:503] Master 127.18.71.126:33209 was elected leader, sending a full tablet report...
I20260809 07:52:26.416141 5734 ts_tablet_manager.cc:1428] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260809 07:52:26.416178 5736 consensus_queue.cc:260] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.417054 5435 heartbeater.cc:503] Master 127.18.71.126:33209 was elected leader, sending a full tablet report...
I20260809 07:52:26.419291 5736 ts_tablet_manager.cc:1428] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
W20260809 07:52:26.440939 5436 tablet.cc:2369] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:52:26.473165 5743 raft_consensus.cc:482] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:26.473425 5743 raft_consensus.cc:504] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.475324 5743 leader_election.cc:288] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2c33c6fa8704eb6804ec7b862967af0 (127.18.71.66:39385), f6495ee2c3ca44faa6c58cdc8e7629d4 (127.18.71.67:43955)
I20260809 07:52:26.480829 5732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.476437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:56393 (local address 127.18.71.67:43955)
0809 07:52:26.476621 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.476626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.476639 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:52:26.476698 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.476701 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.476746 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.476843 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:26.477147 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.477918 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.479326 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.480019 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.480146 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.480203 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.480623 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.480700 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 07:52:26.480852 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.475575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:43955 (local address 127.18.71.65:56393)
0809 07:52:26.476018 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.476032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.476160 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.476854 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.476857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.476873 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:26.477017 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.477023 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.478136 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.478143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.479182 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.479192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.480277 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.480281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.480391 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.480415 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.480599 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.480682 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260809 07:52:26.480928 5741 raft_consensus.cc:482] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:26.481112 5741 raft_consensus.cc:504] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.481091 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.474902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39385 (local address 127.18.71.65:60453)
0809 07:52:26.475201 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.475219 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.475330 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.476264 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.476267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.476285 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:26.476443 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.476472 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.477999 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.478002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.479382 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.479398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.480325 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.480332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.480508 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.480546 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.480635 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.480943 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":117,"threads_started":1}
I20260809 07:52:26.481334 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.475836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:60453 (local address 127.18.71.66:39385)
0809 07:52:26.476022 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.476028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.476047 (+ 19us) server_negotiation.cc:407] Connection header received
0809 07:52:26.476110 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.476114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.476165 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.476249 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:26.476970 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.477872 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.479527 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.480174 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.480296 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.480344 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.481105 (+ 761us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.481183 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 07:52:26.481953 5526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19675690a83f4d3dbd63e3d8db097b08" candidate_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c33c6fa8704eb6804ec7b862967af0" is_pre_election: true
I20260809 07:52:26.482306 5526 raft_consensus.cc:2454] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dfdd9bbb9da4403b515d5ee8e398e53 in term 0.
I20260809 07:52:26.482872 5668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19675690a83f4d3dbd63e3d8db097b08" candidate_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" is_pre_election: true
I20260809 07:52:26.483140 5741 leader_election.cc:288] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2dfdd9bbb9da4403b515d5ee8e398e53 (127.18.71.65:37721), a2c33c6fa8704eb6804ec7b862967af0 (127.18.71.66:39385)
I20260809 07:52:26.483108 5668 raft_consensus.cc:2454] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2dfdd9bbb9da4403b515d5ee8e398e53 in term 0.
I20260809 07:52:26.483538 5315 leader_election.cc:302] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dfdd9bbb9da4403b515d5ee8e398e53, a2c33c6fa8704eb6804ec7b862967af0; no voters:
I20260809 07:52:26.483889 5743 raft_consensus.cc:2781] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:52:26.484017 5743 raft_consensus.cc:482] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:26.484076 5743 raft_consensus.cc:3037] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:26.485507 5743 raft_consensus.cc:504] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.485838 5743 leader_election.cc:288] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [CANDIDATE]: Term 1 election: Requested vote from peers a2c33c6fa8704eb6804ec7b862967af0 (127.18.71.66:39385), f6495ee2c3ca44faa6c58cdc8e7629d4 (127.18.71.67:43955)
I20260809 07:52:26.487360 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.482733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:46063 (local address 127.18.71.65:37721)
0809 07:52:26.482846 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.482850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.483207 (+ 357us) server_negotiation.cc:407] Connection header received
0809 07:52:26.483464 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.483470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.483534 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.483601 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:26.483990 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.484695 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.486170 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.486781 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.486921 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.486971 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.487181 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.487246 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 07:52:26.487375 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.482633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37721 (local address 127.18.71.67:46063)
0809 07:52:26.483095 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.483112 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.483249 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.483638 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.483644 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.483712 (+ 68us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:26.483853 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.483863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.484834 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.484852 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.486039 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.486048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.486922 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.486926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.487042 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.487068 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.487166 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.487239 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301}
I20260809 07:52:26.489176 5668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19675690a83f4d3dbd63e3d8db097b08" candidate_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4"
I20260809 07:52:26.489321 5668 raft_consensus.cc:3037] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:26.489319 5385 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19675690a83f4d3dbd63e3d8db097b08" candidate_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" is_pre_election: true
I20260809 07:52:26.489575 5385 raft_consensus.cc:2379] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f6495ee2c3ca44faa6c58cdc8e7629d4 in current term 1: Already voted for candidate 2dfdd9bbb9da4403b515d5ee8e398e53 in this term.
I20260809 07:52:26.490249 5526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19675690a83f4d3dbd63e3d8db097b08" candidate_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c33c6fa8704eb6804ec7b862967af0"
I20260809 07:52:26.490386 5526 raft_consensus.cc:3037] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:26.490875 5668 raft_consensus.cc:2454] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dfdd9bbb9da4403b515d5ee8e398e53 in term 1.
I20260809 07:52:26.491842 5719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.482658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39385 (local address 127.18.71.67:54871)
0809 07:52:26.483006 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.483017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:26.483077 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:26.484502 (+ 1425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:26.484507 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:26.484529 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:26.484677 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.484688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.489312 (+ 4624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.489318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.490578 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:26.490590 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.491409 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.491413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:26.491551 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.491585 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 07:52:26.491676 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 07:52:26.491745 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":119,"threads_started":1}
I20260809 07:52:26.491887 5526 raft_consensus.cc:2454] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2dfdd9bbb9da4403b515d5ee8e398e53 in term 1.
I20260809 07:52:26.492205 5733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.483298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:54871 (local address 127.18.71.66:39385)
0809 07:52:26.484216 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.484220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.484237 (+ 17us) server_negotiation.cc:407] Connection header received
0809 07:52:26.484286 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.484289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.484329 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.484439 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:26.488550 (+ 4111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.489187 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.490704 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.491268 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.491402 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.491447 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.492015 (+ 568us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.492095 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836}
I20260809 07:52:26.492274 5315 leader_election.cc:302] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2dfdd9bbb9da4403b515d5ee8e398e53, f6495ee2c3ca44faa6c58cdc8e7629d4; no voters:
I20260809 07:52:26.492480 5743 raft_consensus.cc:2781] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:26.492569 5526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19675690a83f4d3dbd63e3d8db097b08" candidate_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c33c6fa8704eb6804ec7b862967af0" is_pre_election: true
I20260809 07:52:26.492751 5526 raft_consensus.cc:2379] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f6495ee2c3ca44faa6c58cdc8e7629d4 in current term 1: Already voted for candidate 2dfdd9bbb9da4403b515d5ee8e398e53 in this term.
I20260809 07:52:26.492728 5743 raft_consensus.cc:686] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [term 1 LEADER]: Becoming Leader. State: Replica: 2dfdd9bbb9da4403b515d5ee8e398e53, State: Running, Role: LEADER
I20260809 07:52:26.493201 5597 leader_election.cc:302] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6495ee2c3ca44faa6c58cdc8e7629d4; no voters: 2dfdd9bbb9da4403b515d5ee8e398e53, a2c33c6fa8704eb6804ec7b862967af0
I20260809 07:52:26.493434 5743 consensus_queue.cc:237] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } }
I20260809 07:52:26.493677 5741 raft_consensus.cc:2726] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 07:52:26.497351 5235 catalog_manager.cc:5495] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2dfdd9bbb9da4403b515d5ee8e398e53 (127.18.71.65). New cstate: current_term: 1 leader_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dfdd9bbb9da4403b515d5ee8e398e53" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 } health_report { overall_health: UNKNOWN } } }
I20260809 07:52:26.520015 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.515881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.18.71.65:37721)
0809 07:52:26.516033 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:26.516061 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:26.516077 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:52:26.516183 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:26.516186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:26.516382 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:26.516540 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:52:26.516960 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.517600 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.518506 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:26.518734 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:26.518840 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:26.519178 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:52:26.519619 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:52:26.519754 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:26.519815 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 07:52:26.519866 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:52:26.528292 5526 raft_consensus.cc:1261] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 1 FOLLOWER]: Refusing update from remote peer 2dfdd9bbb9da4403b515d5ee8e398e53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:26.528409 5668 raft_consensus.cc:1261] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 1 FOLLOWER]: Refusing update from remote peer 2dfdd9bbb9da4403b515d5ee8e398e53: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:26.529085 5743 consensus_queue.cc:1035] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2c33c6fa8704eb6804ec7b862967af0" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 39385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:26.529326 5748 consensus_queue.cc:1035] T 19675690a83f4d3dbd63e3d8db097b08 P 2dfdd9bbb9da4403b515d5ee8e398e53 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6495ee2c3ca44faa6c58cdc8e7629d4" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 43955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:26.538378 5753 mvcc.cc:205] Tried to move back new op lower bound from 7316528932971167744 to 7316528932840714240. Current Snapshot: MvccSnapshot[applied={T|T < 7316528932971167744}]
I20260809 07:52:26.538509 5752 mvcc.cc:205] Tried to move back new op lower bound from 7316528932971167744 to 7316528932840714240. Current Snapshot: MvccSnapshot[applied={T|T < 7316528932971167744}]
I20260809 07:52:26.548039 5236 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:26.548236 5236 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:26.550824 5236 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2a28d3abaf1c4481b10c3042800e24ec: Sending DeleteTablet for 3 replicas of tablet 19675690a83f4d3dbd63e3d8db097b08
I20260809 07:52:26.551610 5364 tablet_service.cc:1514] Processing DeleteTablet for tablet 19675690a83f4d3dbd63e3d8db097b08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:26 UTC) from {username='slave'} at 127.0.0.1:54082
I20260809 07:52:26.551882 5503 tablet_service.cc:1514] Processing DeleteTablet for tablet 19675690a83f4d3dbd63e3d8db097b08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:26 UTC) from {username='slave'} at 127.0.0.1:43624
I20260809 07:52:26.552186 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 5298
I20260809 07:52:26.552408 5646 tablet_service.cc:1514] Processing DeleteTablet for tablet 19675690a83f4d3dbd63e3d8db097b08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:26 UTC) from {username='slave'} at 127.0.0.1:53900
I20260809 07:52:26.554941 5764 tablet_replica.cc:330] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: stopping tablet replica
I20260809 07:52:26.555315 5764 raft_consensus.cc:2229] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:26.555387 5762 tablet_replica.cc:330] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: stopping tablet replica
I20260809 07:52:26.555702 5764 raft_consensus.cc:2258] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:26.555688 5762 raft_consensus.cc:2229] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:26.556044 5762 raft_consensus.cc:2258] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:26.557543 5764 ts_tablet_manager.cc:1905] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:26.557543 5762 ts_tablet_manager.cc:1905] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:26.560571 5762 ts_tablet_manager.cc:1918] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:26.560571 5764 ts_tablet_manager.cc:1918] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:26.560691 5762 log.cc:1196] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/19675690a83f4d3dbd63e3d8db097b08
I20260809 07:52:26.560691 5764 log.cc:1196] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/19675690a83f4d3dbd63e3d8db097b08
I20260809 07:52:26.561072 5764 ts_tablet_manager.cc:1939] T 19675690a83f4d3dbd63e3d8db097b08 P f6495ee2c3ca44faa6c58cdc8e7629d4: Deleting consensus metadata
I20260809 07:52:26.561072 5762 ts_tablet_manager.cc:1939] T 19675690a83f4d3dbd63e3d8db097b08 P a2c33c6fa8704eb6804ec7b862967af0: Deleting consensus metadata
W20260809 07:52:26.561825 5221 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:37721 (error 108)
I20260809 07:52:26.561995 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 5438
I20260809 07:52:26.564167 5729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.563808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:37721 (local address 127.0.0.1:54112)
0809 07:52:26.563972 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.564059 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:37721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 07:52:26.564368 5221 catalog_manager.cc:4584] TS 2dfdd9bbb9da4403b515d5ee8e398e53 (127.18.71.65:37721): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19675690a83f4d3dbd63e3d8db097b08: Network error: Client connection negotiation failed: client connection to 127.18.71.65:37721: connect: Connection refused (error 111)
I20260809 07:52:26.565318 5219 catalog_manager.cc:4841] TS f6495ee2c3ca44faa6c58cdc8e7629d4 (127.18.71.67:43955): tablet 19675690a83f4d3dbd63e3d8db097b08 (table test-table [id=c60835a2516e498d85443e4d50a9259e]) successfully deleted
I20260809 07:52:26.571111 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 5580
I20260809 07:52:26.572693 5729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:26.571273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:39385 (local address 127.0.0.1:43628)
0809 07:52:26.572518 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:26.572587 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.66:39385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1118}
I20260809 07:52:26.578756 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 5203
2026-08-09T07:52:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1077 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:26 dist-test-slave-10pc krb5kdc[5768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:26 dist-test-slave-10pc krb5kdc[5768](info): set up 2 sockets
Aug 09 07:52:26 dist-test-slave-10pc krb5kdc[5768](info): commencing operation
krb5kdc: starting...
W20260809 07:52:27.679872 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:27 dist-test-slave-10pc krb5kdc[5768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:27.805387 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42705
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:42705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:27.919570 5783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:27.919806 5783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:27.919843 5783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:27.923801 5783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:27.923856 5783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:27.923874 5783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:27.923892 5783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:27.923906 5783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:27.929006 5783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:42705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:27.929903 5783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:27.931363 5783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:27.940163 5791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:27.940174 5788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:27.940169 5789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:27.940383 5783 server_base.cc:1034] running on GCE node
I20260809 07:52:27.940941 5783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:27.941995 5783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:27.943162 5783 hybrid_clock.cc:648] HybridClock initialized: now 1786261947943139 us; error 32 us; skew 500 ppm
Aug 09 07:52:27 dist-test-slave-10pc krb5kdc[5768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261947, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:27.946072 5783 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:27.946991 5783 webserver.cc:460] Webserver started at http://127.18.71.126:34535/ using document root <none> and password file <none>
I20260809 07:52:27.947499 5783 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:27.947564 5783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:27.947738 5783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:27.949755 5783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "9902d6e974b94df5a7f007edaa7f90e7"
format_stamp: "Formatted at 2026-08-09 07:52:27 on dist-test-slave-10pc"
server_key: "1f8947e4307ce09922d047b8730dda47"
server_key_iv: "8ac7b8c7d017e5aa6ec835c20bf14b30"
server_key_version: "encryptionkey@0"
I20260809 07:52:27.950259 5783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "9902d6e974b94df5a7f007edaa7f90e7"
format_stamp: "Formatted at 2026-08-09 07:52:27 on dist-test-slave-10pc"
server_key: "1f8947e4307ce09922d047b8730dda47"
server_key_iv: "8ac7b8c7d017e5aa6ec835c20bf14b30"
server_key_version: "encryptionkey@0"
I20260809 07:52:27.953843 5783 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:27.956113 5798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:27.957368 5783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:27.957468 5783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "9902d6e974b94df5a7f007edaa7f90e7"
format_stamp: "Formatted at 2026-08-09 07:52:27 on dist-test-slave-10pc"
server_key: "1f8947e4307ce09922d047b8730dda47"
server_key_iv: "8ac7b8c7d017e5aa6ec835c20bf14b30"
server_key_version: "encryptionkey@0"
I20260809 07:52:27.957561 5783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:27.981585 5783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:27.985039 5783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:27.985209 5783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:27.992295 5783 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:42705
I20260809 07:52:27.992323 5856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:42705 every 8 connection(s)
I20260809 07:52:27.993338 5783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:27.996244 5857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:28.002154 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 5783
I20260809 07:52:28.002151 5857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7: Bootstrap starting.
I20260809 07:52:28.002306 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:28.002640 18717 external_mini_cluster.cc:1428] Setting key 35a36dce1a56cab308fa6d925927f06d
I20260809 07:52:28.004901 5857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:28.005671 5857 log.cc:824] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:28.007540 5857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7: No bootstrap required, opened a new log
I20260809 07:52:28.011000 5857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9902d6e974b94df5a7f007edaa7f90e7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42705 } }
I20260809 07:52:28.011237 5857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:28.011298 5857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9902d6e974b94df5a7f007edaa7f90e7, State: Initialized, Role: FOLLOWER
Aug 09 07:52:28 dist-test-slave-10pc krb5kdc[5768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:28.011768 5857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9902d6e974b94df5a7f007edaa7f90e7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42705 } }
I20260809 07:52:28.011942 5857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:28.012030 5857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:28.012132 5857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:28.013070 5857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9902d6e974b94df5a7f007edaa7f90e7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42705 } }
I20260809 07:52:28.013402 5857 leader_election.cc:302] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9902d6e974b94df5a7f007edaa7f90e7; no voters:
I20260809 07:52:28.013739 5857 leader_election.cc:288] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:28.013836 5862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:28.014062 5862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [term 1 LEADER]: Becoming Leader. State: Replica: 9902d6e974b94df5a7f007edaa7f90e7, State: Running, Role: LEADER
I20260809 07:52:28.014380 5862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9902d6e974b94df5a7f007edaa7f90e7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42705 } }
I20260809 07:52:28.015062 5857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:28.015661 5864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9902d6e974b94df5a7f007edaa7f90e7. Latest consensus state: current_term: 1 leader_uuid: "9902d6e974b94df5a7f007edaa7f90e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9902d6e974b94df5a7f007edaa7f90e7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42705 } } }
I20260809 07:52:28.015800 5864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:28.016175 5863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9902d6e974b94df5a7f007edaa7f90e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9902d6e974b94df5a7f007edaa7f90e7" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42705 } } }
I20260809 07:52:28.016276 5863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:28.016723 5868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:28.018798 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.004061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50056 (local address 127.18.71.126:42705)
0809 07:52:28.004579 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.004595 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.004630 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:52:28.005393 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.005412 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.005780 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.006141 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:28.006996 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.008037 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.008995 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.009482 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.009668 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.012435 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:28.012442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:28.012483 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:28.012513 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:28.014702 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.015264 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.015271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.015277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.015429 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.015758 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.015761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.015763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.016359 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:28.016578 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.017529 (+ 951us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.017844 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":143,"threads_started":1}
I20260809 07:52:28.019924 5868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:28.025898 5868 catalog_manager.cc:1347] Generated new cluster ID: 19a630cf635f4b1b8a04d07865a20a3a
I20260809 07:52:28.025971 5868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:28.037361 5868 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:28.038650 5868 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:28.054896 5868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7: Generated new TSK 0
I20260809 07:52:28.055711 5868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:52:28.087683 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42705
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:28.201632 5884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:28.201885 5884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:28.201926 5884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:28.205996 5884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:28.206055 5884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:28.206138 5884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:52:28.211262 5884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:28.212371 5884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:28.213415 5884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:28.223495 5890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:28.223547 5884 server_base.cc:1034] running on GCE node
W20260809 07:52:28.223507 5892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.223508 5889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:28.224169 5884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:28.224889 5884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:28.226119 5884 hybrid_clock.cc:648] HybridClock initialized: now 1786261948226103 us; error 36 us; skew 500 ppm
Aug 09 07:52:28 dist-test-slave-10pc krb5kdc[5768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:28.233323 5884 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:52:28.234313 5884 webserver.cc:460] Webserver started at http://127.18.71.65:42759/ using document root <none> and password file <none>
I20260809 07:52:28.234915 5884 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:28.235014 5884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:28.235270 5884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:28.237663 5884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "17d9af7a82c34917a3b40033a70f071c"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "cd9be034b73bb7c49d37e1860f9ed79e"
server_key_iv: "cea0fe80ff8e74afd1da7819a7c30496"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.238397 5884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "17d9af7a82c34917a3b40033a70f071c"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "cd9be034b73bb7c49d37e1860f9ed79e"
server_key_iv: "cea0fe80ff8e74afd1da7819a7c30496"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.243806 5884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 07:52:28.246896 5901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.248685 5884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:28.248806 5884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "17d9af7a82c34917a3b40033a70f071c"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "cd9be034b73bb7c49d37e1860f9ed79e"
server_key_iv: "cea0fe80ff8e74afd1da7819a7c30496"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.248914 5884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:28.260054 5884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:28.263563 5884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:28.263753 5884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:28.264340 5884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:28.265203 5884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:28.265252 5884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.265322 5884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:28.265354 5884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.274448 5884 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:40325
I20260809 07:52:28.274473 6018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:40325 every 8 connection(s)
I20260809 07:52:28.275329 5884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
Aug 09 07:52:28 dist-test-slave-10pc krb5kdc[5768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:28.285315 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 5884
I20260809 07:52:28.285467 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:52:28.285812 18717 external_mini_cluster.cc:1428] Setting key e7b1ca1e9d119deeb71dcbac25b4fdb4
I20260809 07:52:28.289093 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.277187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:54377 (local address 127.18.71.126:42705)
0809 07:52:28.277325 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.277329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.278063 (+ 734us) server_negotiation.cc:407] Connection header received
0809 07:52:28.279447 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.279450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.279511 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.279616 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:28.280874 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.281421 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.282582 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.282810 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.282914 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.285408 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:28.285411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:28.285413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:28.285459 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:28.286996 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.287521 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.287525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.287527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.287611 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.287950 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.287954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.287956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.288141 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:28.288250 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.288804 (+ 554us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.288926 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:52:28.290057 6021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.277481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42705 (local address 127.18.71.65:54377)
0809 07:52:28.277912 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:28.277954 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:28.279229 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:28.279802 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:28.279808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:28.280152 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:28.280726 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.280741 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.281553 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.281561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.282436 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.282446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.282967 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.282971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.283134 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.283738 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:28.283754 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:28.285237 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:28.287135 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:28.287143 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:28.287157 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:28.287416 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:28.287706 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:28.287709 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:28.287711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:28.287851 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:28.288250 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:28.288257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:28.288551 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 07:52:28.288758 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 07:52:28.289068 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":112,"threads_started":1}
I20260809 07:52:28.291272 6019 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42705
I20260809 07:52:28.291514 6019 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:28.291988 6019 heartbeater.cc:511] Master 127.18.71.126:42705 requested a full tablet report, sending...
I20260809 07:52:28.293680 5817 ts_manager.cc:194] Registered new tserver with Master: 17d9af7a82c34917a3b40033a70f071c (127.18.71.65:40325)
I20260809 07:52:28.295346 5817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:54377
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:52:28.313709 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42705
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:28.430523 6024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:28.430763 6024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:28.430857 6024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:28.434866 6024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:28.434919 6024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:28.435014 6024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:52:28.440181 6024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:28.441190 6024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:28.442190 6024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:28.452440 6029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.452443 6030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.452443 6032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:28.452683 6024 server_base.cc:1034] running on GCE node
I20260809 07:52:28.453262 6024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:28.453914 6024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:28.455106 6024 hybrid_clock.cc:648] HybridClock initialized: now 1786261948455094 us; error 35 us; skew 500 ppm
Aug 09 07:52:28 dist-test-slave-10pc krb5kdc[5768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:28.457937 6024 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:52:28.458858 6024 webserver.cc:460] Webserver started at http://127.18.71.66:38663/ using document root <none> and password file <none>
I20260809 07:52:28.459455 6024 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:28.459554 6024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:28.459810 6024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:28.462107 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "875cc629f18541b5b25d1766807bd971"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "3d84c1fba8895ac19ebc6587cbc66452"
server_key_iv: "748cacf9e35957755b06d1cf12e08c69"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.462630 6024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "875cc629f18541b5b25d1766807bd971"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "3d84c1fba8895ac19ebc6587cbc66452"
server_key_iv: "748cacf9e35957755b06d1cf12e08c69"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.466382 6024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:28.468698 6041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.470085 6024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:28.470213 6024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "875cc629f18541b5b25d1766807bd971"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "3d84c1fba8895ac19ebc6587cbc66452"
server_key_iv: "748cacf9e35957755b06d1cf12e08c69"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.470309 6024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:28.478924 6024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:28.481705 6024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:28.481899 6024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:28.482479 6024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:28.483264 6024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:28.483310 6024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.483371 6024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:28.483404 6024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.492889 6024 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:37719
I20260809 07:52:28.492916 6161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:37719 every 8 connection(s)
I20260809 07:52:28.493747 6024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:28.500765 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 6024
I20260809 07:52:28.500913 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:52:28.501235 18717 external_mini_cluster.cc:1428] Setting key 17aeebd182a370ebb4964fade1ec4e78
Aug 09 07:52:28 dist-test-slave-10pc krb5kdc[5768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:28.508608 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.495568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:55401 (local address 127.18.71.126:42705)
0809 07:52:28.495719 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.495723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.496520 (+ 797us) server_negotiation.cc:407] Connection header received
0809 07:52:28.497911 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.497914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.497975 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.498083 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:28.499281 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.499825 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.501138 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.501433 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.501544 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.504397 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:28.504401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:28.504404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:28.504437 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:28.506194 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.506886 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.506890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.506892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.506986 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.507366 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.507371 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.507373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.507581 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:28.507702 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.508254 (+ 552us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.508390 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 07:52:28.509521 6164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.495890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42705 (local address 127.18.71.66:55401)
0809 07:52:28.496352 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:28.496395 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:28.497690 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:28.498288 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:28.498295 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:28.498648 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:28.499127 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.499141 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.499954 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.499960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.500976 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.501010 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.501589 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.501593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.501801 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.502480 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:28.502498 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:28.504215 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:28.506329 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:28.506335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:28.506352 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:28.506769 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:28.507095 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:28.507098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:28.507100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:28.507256 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:28.507704 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:28.507712 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:28.507997 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 07:52:28.508209 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 07:52:28.508597 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":122,"threads_started":1}
I20260809 07:52:28.510808 6162 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42705
I20260809 07:52:28.511122 6162 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:28.511662 6162 heartbeater.cc:511] Master 127.18.71.126:42705 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:52:28.512856 5817 ts_manager.cc:194] Registered new tserver with Master: 875cc629f18541b5b25d1766807bd971 (127.18.71.66:37719)
I20260809 07:52:28.513447 5817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:55401
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:52:28.531052 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42705
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:28.645148 6167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:28.645381 6167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:28.645421 6167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:28.649401 6167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:28.649454 6167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:28.649556 6167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:52:28.654726 6167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:42705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:28.655735 6167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:28.656807 6167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:28.666651 6172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.666669 6173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:28.666613 6175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:28.666857 6167 server_base.cc:1034] running on GCE node
I20260809 07:52:28.667326 6167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:28.667975 6167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:28.669134 6167 hybrid_clock.cc:648] HybridClock initialized: now 1786261948669105 us; error 45 us; skew 500 ppm
Aug 09 07:52:28 dist-test-slave-10pc krb5kdc[5768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:28.671838 6167 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:52:28.672816 6167 webserver.cc:460] Webserver started at http://127.18.71.67:40441/ using document root <none> and password file <none>
I20260809 07:52:28.673388 6167 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:28.673480 6167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:28.673722 6167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:28.675851 6167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "fbff01a4af669d50b35b89ee5a80667e"
server_key_iv: "5dfd9eb2a731b889d9fff5c61a6da671"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.676429 6167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "fbff01a4af669d50b35b89ee5a80667e"
server_key_iv: "5dfd9eb2a731b889d9fff5c61a6da671"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.680171 6167 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:28.682617 6182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.683771 6167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:52:28.683911 6167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3"
format_stamp: "Formatted at 2026-08-09 07:52:28 on dist-test-slave-10pc"
server_key: "fbff01a4af669d50b35b89ee5a80667e"
server_key_iv: "5dfd9eb2a731b889d9fff5c61a6da671"
server_key_version: "encryptionkey@0"
I20260809 07:52:28.684023 6167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:28.699970 6167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:28.702905 6167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:28.703120 6167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:28.703733 6167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:28.704612 6167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:28.704685 6167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.704758 6167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:28.704811 6167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:28.713865 6167 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:33167
I20260809 07:52:28.713896 6305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:33167 every 8 connection(s)
I20260809 07:52:28.714848 6167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:28.718225 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 6167
I20260809 07:52:28.718331 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:52:28.718655 18717 external_mini_cluster.cc:1428] Setting key d1d52b8e854cb77a9971a3c470aa4c54
Aug 09 07:52:28 dist-test-slave-10pc krb5kdc[5768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261948, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:28.728914 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.716851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:36849 (local address 127.18.71.126:42705)
0809 07:52:28.716991 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.716995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.717844 (+ 849us) server_negotiation.cc:407] Connection header received
0809 07:52:28.719482 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.719485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.719539 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.719624 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:28.720897 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.721471 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.722711 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.722942 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.723056 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.725578 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:28.725581 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:28.725583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:28.725607 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:28.726852 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.727363 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.727367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.727369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.727455 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.727802 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.727807 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.727809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.727988 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:28.728099 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.728630 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.728755 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:52:28.729712 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.717246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:42705 (local address 127.18.71.67:36849)
0809 07:52:28.717692 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:28.717734 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:28.719250 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:28.719819 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:28.719826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:28.720180 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:28.720749 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.720763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.721621 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.721624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.722565 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.722575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.723098 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.723103 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.723279 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.723894 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:28.723911 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:28.725404 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:28.726973 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:28.726979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:28.726994 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:28.727255 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:28.727550 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:28.727553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:28.727555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:28.727702 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:28.728100 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:28.728108 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:28.728348 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 07:52:28.728585 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 07:52:28.728917 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260809 07:52:28.730785 6307 heartbeater.cc:347] Connected to a master server at 127.18.71.126:42705
I20260809 07:52:28.731004 6307 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:28.731528 6307 heartbeater.cc:511] Master 127.18.71.126:42705 requested a full tablet report, sending...
I20260809 07:52:28.732694 5817 ts_manager.cc:194] Registered new tserver with Master: d129d8cffb9547bf9bc6bb96b2e3efb3 (127.18.71.67:33167)
I20260809 07:52:28.733239 5817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:36849
I20260809 07:52:28.744817 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:52:28.754734 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.746969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.18.71.126:42705)
0809 07:52:28.747094 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.747097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.747277 (+ 180us) server_negotiation.cc:407] Connection header received
0809 07:52:28.747383 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.747385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.747425 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.747492 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:28.748185 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.748767 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.749561 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.749765 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.749836 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.751429 (+ 1593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:28.751432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:28.751434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:28.751453 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:28.752880 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.753361 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.753365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.753367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.753456 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:28.753800 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:28.753803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:28.753805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:28.754016 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:28.754192 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.754467 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.754561 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 07:52:28.757652 5817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:52:28.760818 5817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:52:28.775799 6322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.770138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60338 (local address 127.18.71.67:33167)
0809 07:52:28.770597 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.770610 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.770634 (+ 24us) server_negotiation.cc:407] Connection header received
0809 07:52:28.770714 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.770721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.770922 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.771087 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:28.771473 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.772373 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.774142 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.774892 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.775046 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.775164 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.775536 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.775645 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":140,"threads_started":1}
I20260809 07:52:28.775806 6318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.769340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40325 (local address 127.0.0.1:55728)
0809 07:52:28.769694 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:28.769746 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:28.769875 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:28.771097 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:28.771104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:28.771142 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:28.771303 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.771310 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.772822 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.772826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.774070 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.774080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.775196 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.775200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.775344 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.775380 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 07:52:28.775440 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 07:52:28.775625 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":92,"spinlock_wait_cycles":768,"thread_start_us":72,"threads_started":1}
I20260809 07:52:28.776252 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.770042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:37719 (local address 127.0.0.1:52740)
0809 07:52:28.770958 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:28.770968 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:28.771132 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:28.771658 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:28.771661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:28.771680 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:28.771761 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.771766 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.773379 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.773382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.774495 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.774506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.775829 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.775836 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.775932 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.775964 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 07:52:28.776058 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 07:52:28.776123 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":32,"thread_start_us":96,"threads_started":1}
I20260809 07:52:28.775790 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.769721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:33167 (local address 127.0.0.1:60338)
0809 07:52:28.770273 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:28.770287 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:28.770373 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:28.771282 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:28.771285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:28.771297 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:28.771378 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.771383 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.772560 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.772564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.774018 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:28.774030 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.775221 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.775224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:28.775319 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.775354 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 07:52:28.775472 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 07:52:28.775627 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":82,"thread_start_us":85,"threads_started":1}
I20260809 07:52:28.776548 6323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.770469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.18.71.66:37719)
0809 07:52:28.770842 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.770850 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.771067 (+ 217us) server_negotiation.cc:407] Connection header received
0809 07:52:28.771299 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.771303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.771497 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.771653 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:28.771876 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.773177 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.774635 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.775508 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.775629 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.775765 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.776118 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.776218 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":108,"threads_started":1}
I20260809 07:52:28.777272 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:28.769482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.18.71.65:40325)
0809 07:52:28.770671 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:28.770685 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:28.770705 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:52:28.770768 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:28.770773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:28.770921 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:28.771087 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:28.771421 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.772657 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.774219 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:28.775039 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:28.776830 (+ 1791us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:28.776965 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:28.777012 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 07:52:28.777097 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":87,"threads_started":1}
I20260809 07:52:28.779158 6091 tablet_service.cc:1467] Processing CreateTablet for tablet 6b821202293f4ed29bb945b44dbb31e0 (DEFAULT_TABLE table=test-table [id=be1c62bc37db49ad83e1fc591504422f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:28.779311 6236 tablet_service.cc:1467] Processing CreateTablet for tablet 6b821202293f4ed29bb945b44dbb31e0 (DEFAULT_TABLE table=test-table [id=be1c62bc37db49ad83e1fc591504422f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:28.780037 6091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b821202293f4ed29bb945b44dbb31e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:28.779736 5950 tablet_service.cc:1467] Processing CreateTablet for tablet 6b821202293f4ed29bb945b44dbb31e0 (DEFAULT_TABLE table=test-table [id=be1c62bc37db49ad83e1fc591504422f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:28.780313 6236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b821202293f4ed29bb945b44dbb31e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:28.780646 5950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b821202293f4ed29bb945b44dbb31e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:28.786367 6324 tablet_bootstrap.cc:492] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Bootstrap starting.
I20260809 07:52:28.787878 6326 tablet_bootstrap.cc:492] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Bootstrap starting.
I20260809 07:52:28.788861 6327 tablet_bootstrap.cc:492] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c: Bootstrap starting.
I20260809 07:52:28.789566 6324 tablet_bootstrap.cc:654] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:28.789893 6326 tablet_bootstrap.cc:654] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:28.790494 6324 log.cc:824] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:28.791023 6327 tablet_bootstrap.cc:654] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:28.791108 6326 log.cc:824] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:28.791698 6327 log.cc:824] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:28.793058 6326 tablet_bootstrap.cc:492] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: No bootstrap required, opened a new log
I20260809 07:52:28.793136 6324 tablet_bootstrap.cc:492] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: No bootstrap required, opened a new log
I20260809 07:52:28.793273 6326 ts_tablet_manager.cc:1397] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:52:28.793327 6324 ts_tablet_manager.cc:1397] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:52:28.793505 6327 tablet_bootstrap.cc:492] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c: No bootstrap required, opened a new log
I20260809 07:52:28.793679 6327 ts_tablet_manager.cc:1397] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:52:28.796240 6326 raft_consensus.cc:348] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:28.796512 6326 raft_consensus.cc:374] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:28.796571 6326 raft_consensus.cc:729] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d129d8cffb9547bf9bc6bb96b2e3efb3, State: Initialized, Role: FOLLOWER
I20260809 07:52:28.797043 6326 consensus_queue.cc:260] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:28.797326 6324 raft_consensus.cc:348] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:28.797595 6324 raft_consensus.cc:374] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:28.797662 6324 raft_consensus.cc:729] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 875cc629f18541b5b25d1766807bd971, State: Initialized, Role: FOLLOWER
I20260809 07:52:28.797840 6327 raft_consensus.cc:348] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:28.798089 6326 ts_tablet_manager.cc:1428] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260809 07:52:28.798102 6327 raft_consensus.cc:374] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:28.798157 6327 raft_consensus.cc:729] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d9af7a82c34917a3b40033a70f071c, State: Initialized, Role: FOLLOWER
I20260809 07:52:28.798125 6324 consensus_queue.cc:260] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:28.798426 6307 heartbeater.cc:503] Master 127.18.71.126:42705 was elected leader, sending a full tablet report...
I20260809 07:52:28.798544 6327 consensus_queue.cc:260] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:28.799103 6324 ts_tablet_manager.cc:1428] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 07:52:28.799221 6019 heartbeater.cc:503] Master 127.18.71.126:42705 was elected leader, sending a full tablet report...
I20260809 07:52:28.799422 6162 heartbeater.cc:503] Master 127.18.71.126:42705 was elected leader, sending a full tablet report...
I20260809 07:52:28.799517 6327 ts_tablet_manager.cc:1428] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
W20260809 07:52:28.965864 6308 tablet.cc:2369] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:52:28.994921 6163 tablet.cc:2369] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:52:29.026714 6020 tablet.cc:2369] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:52:29.104146 6331 raft_consensus.cc:482] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:29.104403 6331 raft_consensus.cc:504] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:29.105718 6331 leader_election.cc:288] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17d9af7a82c34917a3b40033a70f071c (127.18.71.65:40325), 875cc629f18541b5b25d1766807bd971 (127.18.71.66:37719)
I20260809 07:52:29.110930 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.106273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:60223 (local address 127.18.71.65:40325)
0809 07:52:29.106454 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:29.106461 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:29.106503 (+ 42us) server_negotiation.cc:407] Connection header received
0809 07:52:29.106555 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:29.106557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:29.106624 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.106701 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:29.107068 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.107859 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.109564 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.110155 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.110296 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.110350 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:29.110726 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 07:52:29.110801 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 07:52:29.110930 6323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.106541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:54365 (local address 127.18.71.66:37719)
0809 07:52:29.106716 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:29.106721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:29.106879 (+ 158us) server_negotiation.cc:407] Connection header received
0809 07:52:29.107030 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:29.107034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:29.107089 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.107170 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:29.107456 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.108108 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.109586 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.110218 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.110323 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.110368 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:29.110726 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 07:52:29.110801 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 07:52:29.110935 6309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.106082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40325 (local address 127.18.71.67:60223)
0809 07:52:29.106223 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:29.106237 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:29.106375 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:29.106761 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:29.106764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:29.106789 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:29.106943 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.106950 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.107991 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.107993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.109389 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.109400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.110299 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.110303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.110525 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.110582 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 07:52:29.110670 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 07:52:29.110766 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":159}
I20260809 07:52:29.110944 6334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.106394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:37719 (local address 127.18.71.67:54365)
0809 07:52:29.106779 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:29.106792 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:29.106892 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:29.107217 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:29.107219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:29.107236 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:29.107336 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.107340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.108210 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.108213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.109468 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.109478 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.110323 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.110326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.110512 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.110603 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 07:52:29.110672 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 07:52:29.110766 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":131,"thread_start_us":143,"threads_started":1}
I20260809 07:52:29.111738 6113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b821202293f4ed29bb945b44dbb31e0" candidate_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875cc629f18541b5b25d1766807bd971" is_pre_election: true
I20260809 07:52:29.112011 6113 raft_consensus.cc:2454] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d129d8cffb9547bf9bc6bb96b2e3efb3 in term 0.
I20260809 07:52:29.112493 6185 leader_election.cc:302] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 875cc629f18541b5b25d1766807bd971, d129d8cffb9547bf9bc6bb96b2e3efb3; no voters:
I20260809 07:52:29.112571 5972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b821202293f4ed29bb945b44dbb31e0" candidate_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17d9af7a82c34917a3b40033a70f071c" is_pre_election: true
I20260809 07:52:29.112769 6331 raft_consensus.cc:2781] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:52:29.112857 6331 raft_consensus.cc:482] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:29.112933 6331 raft_consensus.cc:3037] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:29.112908 5972 raft_consensus.cc:2454] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d129d8cffb9547bf9bc6bb96b2e3efb3 in term 0.
I20260809 07:52:29.114065 6331 raft_consensus.cc:504] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:29.114416 6331 leader_election.cc:288] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [CANDIDATE]: Term 1 election: Requested vote from peers 17d9af7a82c34917a3b40033a70f071c (127.18.71.65:40325), 875cc629f18541b5b25d1766807bd971 (127.18.71.66:37719)
I20260809 07:52:29.114825 5972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b821202293f4ed29bb945b44dbb31e0" candidate_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17d9af7a82c34917a3b40033a70f071c"
I20260809 07:52:29.114945 5972 raft_consensus.cc:3037] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:29.114961 6113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b821202293f4ed29bb945b44dbb31e0" candidate_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "875cc629f18541b5b25d1766807bd971"
I20260809 07:52:29.115099 6113 raft_consensus.cc:3037] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:29.116200 5972 raft_consensus.cc:2454] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d129d8cffb9547bf9bc6bb96b2e3efb3 in term 1.
I20260809 07:52:29.116375 6113 raft_consensus.cc:2454] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d129d8cffb9547bf9bc6bb96b2e3efb3 in term 1.
I20260809 07:52:29.116565 6185 leader_election.cc:302] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17d9af7a82c34917a3b40033a70f071c, d129d8cffb9547bf9bc6bb96b2e3efb3; no voters:
I20260809 07:52:29.116751 6331 raft_consensus.cc:2781] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:29.117018 6331 raft_consensus.cc:686] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 1 LEADER]: Becoming Leader. State: Replica: d129d8cffb9547bf9bc6bb96b2e3efb3, State: Running, Role: LEADER
I20260809 07:52:29.117354 6331 consensus_queue.cc:237] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } }
I20260809 07:52:29.120501 5817 catalog_manager.cc:5495] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 reported cstate change: term changed from 0 to 1, leader changed from <none> to d129d8cffb9547bf9bc6bb96b2e3efb3 (127.18.71.67). New cstate: current_term: 1 leader_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d129d8cffb9547bf9bc6bb96b2e3efb3" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 33167 } health_report { overall_health: HEALTHY } } }
I20260809 07:52:29.137445 6322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.134000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60350 (local address 127.18.71.67:33167)
0809 07:52:29.134127 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:29.134130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:29.134143 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:52:29.134244 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:29.134246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:29.134296 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.134388 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:52:29.134636 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.135118 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.135958 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.136162 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.136234 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.136593 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:52:29.137018 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:52:29.137132 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:29.137286 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 07:52:29.137320 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:52:29.143545 6113 raft_consensus.cc:1261] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 1 FOLLOWER]: Refusing update from remote peer d129d8cffb9547bf9bc6bb96b2e3efb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:29.143667 5972 raft_consensus.cc:1261] T 6b821202293f4ed29bb945b44dbb31e0 P 17d9af7a82c34917a3b40033a70f071c [term 1 FOLLOWER]: Refusing update from remote peer d129d8cffb9547bf9bc6bb96b2e3efb3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:29.144383 6335 consensus_queue.cc:1035] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "875cc629f18541b5b25d1766807bd971" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:29.144598 6331 consensus_queue.cc:1035] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17d9af7a82c34917a3b40033a70f071c" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:29.152136 6338 mvcc.cc:205] Tried to move back new op lower bound from 7316528943684075520 to 7316528943587254272. Current Snapshot: MvccSnapshot[applied={T|T < 7316528943684075520}]
I20260809 07:52:29.155279 6340 mvcc.cc:205] Tried to move back new op lower bound from 7316528943684075520 to 7316528943587254272. Current Snapshot: MvccSnapshot[applied={T|T < 7316528943684075520}]
I20260809 07:52:29.162288 5817 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:29.162495 5817 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:29.165277 5817 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9902d6e974b94df5a7f007edaa7f90e7: Sending DeleteTablet for 3 replicas of tablet 6b821202293f4ed29bb945b44dbb31e0
I20260809 07:52:29.166167 6236 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b821202293f4ed29bb945b44dbb31e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:29 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:60338
I20260809 07:52:29.166340 5950 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b821202293f4ed29bb945b44dbb31e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:29 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:55728
I20260809 07:52:29.166702 6349 tablet_replica.cc:330] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: stopping tablet replica
I20260809 07:52:29.166860 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 5884
I20260809 07:52:29.166852 6091 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b821202293f4ed29bb945b44dbb31e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:29 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:52740
I20260809 07:52:29.167038 6349 raft_consensus.cc:2229] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:52:29.167456 6349 raft_consensus.cc:2258] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:29.169385 6349 ts_tablet_manager.cc:1905] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:29.170126 6351 tablet_replica.cc:330] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: stopping tablet replica
I20260809 07:52:29.170562 6351 raft_consensus.cc:2229] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:29.170985 6351 raft_consensus.cc:2258] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:29.172531 6349 ts_tablet_manager.cc:1918] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:29.172641 6349 log.cc:1196] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/6b821202293f4ed29bb945b44dbb31e0
I20260809 07:52:29.172832 6351 ts_tablet_manager.cc:1905] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:29.173046 6349 ts_tablet_manager.cc:1939] T 6b821202293f4ed29bb945b44dbb31e0 P d129d8cffb9547bf9bc6bb96b2e3efb3: Deleting consensus metadata
I20260809 07:52:29.176433 6351 ts_tablet_manager.cc:1918] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:29.176589 6351 log.cc:1196] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/6b821202293f4ed29bb945b44dbb31e0
I20260809 07:52:29.176920 6351 ts_tablet_manager.cc:1939] T 6b821202293f4ed29bb945b44dbb31e0 P 875cc629f18541b5b25d1766807bd971: Deleting consensus metadata
I20260809 07:52:29.178360 5801 catalog_manager.cc:4841] TS 875cc629f18541b5b25d1766807bd971 (127.18.71.66:37719): tablet 6b821202293f4ed29bb945b44dbb31e0 (table test-table [id=be1c62bc37db49ad83e1fc591504422f]) successfully deleted
W20260809 07:52:29.180768 5801 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:40325 (error 108)
I20260809 07:52:29.180886 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 6024
I20260809 07:52:29.187096 5802 catalog_manager.cc:4841] TS d129d8cffb9547bf9bc6bb96b2e3efb3 (127.18.71.67:33167): tablet 6b821202293f4ed29bb945b44dbb31e0 (table test-table [id=be1c62bc37db49ad83e1fc591504422f]) successfully deleted
I20260809 07:52:29.189235 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.188907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40325 (local address 127.0.0.1:55732)
0809 07:52:29.189062 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:29.189142 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 07:52:29.189440 5801 catalog_manager.cc:4584] TS 17d9af7a82c34917a3b40033a70f071c (127.18.71.65:40325): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6b821202293f4ed29bb945b44dbb31e0: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40325: connect: Connection refused (error 111)
I20260809 07:52:29.194106 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 6167
I20260809 07:52:29.201287 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 5783
2026-08-09T07:52:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2622 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T07:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:29Z Disabled control of system clock
I20260809 07:52:29.236964 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:35521
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:35521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 07:52:29.349735 6357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:29.350023 6357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:29.350059 6357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:29.354058 6357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:29.354125 6357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:29.354143 6357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:29.354161 6357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:29.354175 6357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:29.358916 6357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:35521
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:35521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:29.359967 6357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:29.361356 6357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:29.369769 6365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:29.369763 6363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.369817 6357 server_base.cc:1034] running on GCE node
W20260809 07:52:29.369764 6362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.370633 6357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:29.371737 6357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:29.372895 6357 hybrid_clock.cc:648] HybridClock initialized: now 1786261949372852 us; error 53 us; skew 500 ppm
I20260809 07:52:29.374858 6357 webserver.cc:460] Webserver started at http://127.18.71.126:44099/ using document root <none> and password file <none>
I20260809 07:52:29.375444 6357 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:29.375509 6357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:29.375764 6357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:29.377872 6357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "47cc2ae1861a413aaf2fb481adca1727"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "ef35c95fdb3a3245d16b8af8304c4bcc"
server_key_iv: "25704097e77bfde27381e0f08a97d662"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.378459 6357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "47cc2ae1861a413aaf2fb481adca1727"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "ef35c95fdb3a3245d16b8af8304c4bcc"
server_key_iv: "25704097e77bfde27381e0f08a97d662"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.382148 6357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 07:52:29.384588 6371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.385833 6357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:29.385962 6357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "47cc2ae1861a413aaf2fb481adca1727"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "ef35c95fdb3a3245d16b8af8304c4bcc"
server_key_iv: "25704097e77bfde27381e0f08a97d662"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.386068 6357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:29.401711 6357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:29.402418 6357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:29.402621 6357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:29.409421 6357 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:35521
I20260809 07:52:29.409428 6426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:35521 every 8 connection(s)
I20260809 07:52:29.410377 6357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:29.412926 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 6357
I20260809 07:52:29.413069 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:29.413535 18717 external_mini_cluster.cc:1428] Setting key c51fe375f110186ffb41a0d21a6661e6
I20260809 07:52:29.414073 6427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:29.420656 6427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727: Bootstrap starting.
I20260809 07:52:29.422909 6427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:29.423249 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.415110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42418 (local address 127.18.71.126:35521)
0809 07:52:29.415767 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:29.415782 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:29.415817 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:52:29.416899 (+ 1082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:29.416919 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:29.417308 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.417660 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:29.418086 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.419032 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.419932 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.420412 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.420645 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.421542 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:29.421548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:29.421566 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:29.421937 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:29.421990 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:29.422010 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:29.422132 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:29.422252 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 07:52:29.422506 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":149,"threads_started":1}
I20260809 07:52:29.423779 6427 log.cc:824] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:29.426025 6427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727: No bootstrap required, opened a new log
I20260809 07:52:29.429976 6427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cc2ae1861a413aaf2fb481adca1727" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 35521 } }
I20260809 07:52:29.430295 6427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:29.430370 6427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47cc2ae1861a413aaf2fb481adca1727, State: Initialized, Role: FOLLOWER
I20260809 07:52:29.430783 6427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cc2ae1861a413aaf2fb481adca1727" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 35521 } }
I20260809 07:52:29.430904 6427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:29.430948 6427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:29.431017 6427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:29.431833 6427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cc2ae1861a413aaf2fb481adca1727" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 35521 } }
I20260809 07:52:29.432132 6427 leader_election.cc:302] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47cc2ae1861a413aaf2fb481adca1727; no voters:
I20260809 07:52:29.432397 6427 leader_election.cc:288] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:29.432554 6434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:29.432816 6434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [term 1 LEADER]: Becoming Leader. State: Replica: 47cc2ae1861a413aaf2fb481adca1727, State: Running, Role: LEADER
I20260809 07:52:29.433125 6434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cc2ae1861a413aaf2fb481adca1727" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 35521 } }
I20260809 07:52:29.433682 6427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:29.434890 6436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47cc2ae1861a413aaf2fb481adca1727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cc2ae1861a413aaf2fb481adca1727" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 35521 } } }
I20260809 07:52:29.434878 6437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47cc2ae1861a413aaf2fb481adca1727. Latest consensus state: current_term: 1 leader_uuid: "47cc2ae1861a413aaf2fb481adca1727" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47cc2ae1861a413aaf2fb481adca1727" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 35521 } } }
I20260809 07:52:29.435022 6436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:29.435022 6437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:29.435456 6444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:29.438616 6444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:29.444430 6444 catalog_manager.cc:1347] Generated new cluster ID: b6aeafca8e3344ff824f382e38bd996d
I20260809 07:52:29.444516 6444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:29.464653 6444 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:29.465852 6444 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:29.472703 6444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727: Generated new TSK 0
I20260809 07:52:29.473522 6444 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:29.485766 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:35521
--builtin_ntp_servers=127.18.71.84:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 07:52:29.601848 6454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:29.602138 6454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:29.602207 6454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:29.606166 6454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:29.606246 6454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:29.606336 6454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:52:29.610946 6454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:29.612120 6454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:29.613607 6454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:29.623198 6464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.623310 6454 server_base.cc:1034] running on GCE node
W20260809 07:52:29.623238 6460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:29.623476 6459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.623879 6454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:29.624579 6454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:29.625747 6454 hybrid_clock.cc:648] HybridClock initialized: now 1786261949625725 us; error 36 us; skew 500 ppm
I20260809 07:52:29.627653 6454 webserver.cc:460] Webserver started at http://127.18.71.65:41881/ using document root <none> and password file <none>
I20260809 07:52:29.628330 6454 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:29.628398 6454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:29.628640 6454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:29.630698 6454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "691d66335d3a4a748312da6682a10d0e"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "f3f8384c7aa5c7d2c59563f98887ce09"
server_key_iv: "fc5650bc5689a6805f655d16a431d60a"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.631279 6454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "691d66335d3a4a748312da6682a10d0e"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "f3f8384c7aa5c7d2c59563f98887ce09"
server_key_iv: "fc5650bc5689a6805f655d16a431d60a"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.635154 6454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:29.637672 6470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.638875 6454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:52:29.639019 6454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "691d66335d3a4a748312da6682a10d0e"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "f3f8384c7aa5c7d2c59563f98887ce09"
server_key_iv: "fc5650bc5689a6805f655d16a431d60a"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.639139 6454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:29.648582 6454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:29.649194 6454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:29.649385 6454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:29.650053 6454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:29.650966 6454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:29.651036 6454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.651105 6454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:29.651165 6454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.661178 6454 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:43901
I20260809 07:52:29.661212 6590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:43901 every 8 connection(s)
I20260809 07:52:29.662082 6454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:29.663403 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 6454
I20260809 07:52:29.663576 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:52:29.663954 18717 external_mini_cluster.cc:1428] Setting key d9d21266508fedf8efbf49d3a2ade423
I20260809 07:52:29.666232 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:35521
--builtin_ntp_servers=127.18.71.84:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 07:52:29.672515 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.664109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:48627 (local address 127.18.71.126:35521)
0809 07:52:29.664274 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:29.664280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:29.665600 (+ 1320us) server_negotiation.cc:407] Connection header received
0809 07:52:29.667248 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:29.667252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:29.667308 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.667418 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:29.668667 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.669181 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.670408 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.670639 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.670762 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.671946 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:29.671949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:29.671952 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:29.671963 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:29.671982 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:29.671991 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:29.672098 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:29.672265 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 07:52:29.672347 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 07:52:29.673380 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.664637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:35521 (local address 127.18.71.65:48627)
0809 07:52:29.665436 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:29.665496 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:29.667027 (+ 1531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:29.667601 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:29.667608 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:29.667724 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:29.668462 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.668513 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.669313 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.669316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.670272 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.670281 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.670807 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.670811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.670981 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.671675 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:29.671676 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:29.671717 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:29.671780 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:29.672102 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:29.672114 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:29.672123 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 07:52:29.672264 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 07:52:29.672605 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":154,"threads_started":1}
I20260809 07:52:29.674678 6591 heartbeater.cc:347] Connected to a master server at 127.18.71.126:35521
I20260809 07:52:29.674929 6591 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:29.675458 6591 heartbeater.cc:511] Master 127.18.71.126:35521 requested a full tablet report, sending...
I20260809 07:52:29.677289 6389 ts_manager.cc:194] Registered new tserver with Master: 691d66335d3a4a748312da6682a10d0e (127.18.71.65:43901)
I20260809 07:52:29.679040 6389 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.65:48627
W20260809 07:52:29.782258 6594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:29.782509 6594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:29.782547 6594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:29.786576 6594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:29.786631 6594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:29.786682 6594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:52:29.791261 6594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:29.792250 6594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:29.793771 6594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:29.803167 6599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:29.803190 6600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:29.803167 6602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.803653 6594 server_base.cc:1034] running on GCE node
I20260809 07:52:29.804057 6594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:29.804735 6594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:29.805920 6594 hybrid_clock.cc:648] HybridClock initialized: now 1786261949805891 us; error 41 us; skew 500 ppm
I20260809 07:52:29.807690 6594 webserver.cc:460] Webserver started at http://127.18.71.66:34647/ using document root <none> and password file <none>
I20260809 07:52:29.808270 6594 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:29.808367 6594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:29.808655 6594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:29.810683 6594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "e5c9fbd589634cb5b49b87730d9a58c7"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "171627f55a998332b8cae987544e302d"
server_key_iv: "cb54e26b6f97b2a84f4d4d2f6bebe07a"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.811273 6594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "e5c9fbd589634cb5b49b87730d9a58c7"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "171627f55a998332b8cae987544e302d"
server_key_iv: "cb54e26b6f97b2a84f4d4d2f6bebe07a"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.814991 6594 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:52:29.817404 6608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.818549 6594 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:29.818698 6594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "e5c9fbd589634cb5b49b87730d9a58c7"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "171627f55a998332b8cae987544e302d"
server_key_iv: "cb54e26b6f97b2a84f4d4d2f6bebe07a"
server_key_version: "encryptionkey@0"
I20260809 07:52:29.818813 6594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:29.835942 6594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:29.836747 6594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:29.836951 6594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:29.837554 6594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:29.838542 6594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:29.838615 6594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.838681 6594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:29.838725 6594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:29.848320 6594 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:37995
I20260809 07:52:29.848356 6729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:37995 every 8 connection(s)
I20260809 07:52:29.849326 6594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:29.853948 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 6594
I20260809 07:52:29.854110 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:52:29.854462 18717 external_mini_cluster.cc:1428] Setting key 3d3c0ddf70b3a91892e0c3ad7e641a07
I20260809 07:52:29.856731 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:35521
--builtin_ntp_servers=127.18.71.84:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 07:52:29.860021 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.851278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:42107 (local address 127.18.71.126:35521)
0809 07:52:29.851432 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:29.851437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:29.852629 (+ 1192us) server_negotiation.cc:407] Connection header received
0809 07:52:29.854087 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:29.854091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:29.854156 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:29.854253 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:29.855284 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.856015 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.857818 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.858034 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.858122 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.859451 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:29.859457 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:29.859463 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:29.859480 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:29.859507 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:29.859519 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:29.859611 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:29.859757 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 07:52:29.859824 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:52:29.860915 6732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:29.851748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:35521 (local address 127.18.71.66:42107)
0809 07:52:29.852418 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:29.852512 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:29.853852 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:29.854435 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:29.854442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:29.854559 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:29.855125 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.855144 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.856172 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.856176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.857687 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:29.857702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:29.858208 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:29.858215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:29.858456 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:29.859185 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:29.859187 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:29.859229 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:29.859293 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:29.859621 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:29.859628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:29.859632 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 07:52:29.859727 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 07:52:29.859944 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":112,"threads_started":1}
I20260809 07:52:29.862159 6730 heartbeater.cc:347] Connected to a master server at 127.18.71.126:35521
I20260809 07:52:29.862406 6730 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:29.862926 6730 heartbeater.cc:511] Master 127.18.71.126:35521 requested a full tablet report, sending...
I20260809 07:52:29.864041 6389 ts_manager.cc:194] Registered new tserver with Master: e5c9fbd589634cb5b49b87730d9a58c7 (127.18.71.66:37995)
I20260809 07:52:29.864967 6389 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.66:42107
W20260809 07:52:29.974162 6733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:29.974402 6733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:29.974438 6733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:29.978425 6733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:29.978473 6733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:29.978523 6733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:52:29.982924 6733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:35521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:29.984038 6733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:29.985535 6733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:29.995525 6739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:29.995497 6742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.995774 6733 server_base.cc:1034] running on GCE node
W20260809 07:52:29.995882 6740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:29.996280 6733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:29.996984 6733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:29.998171 6733 hybrid_clock.cc:648] HybridClock initialized: now 1786261949998136 us; error 47 us; skew 500 ppm
I20260809 07:52:30.000047 6733 webserver.cc:460] Webserver started at http://127.18.71.67:35993/ using document root <none> and password file <none>
I20260809 07:52:30.000710 6733 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:30.000806 6733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:30.001050 6733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:30.003391 6733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "7f749dcc5135472ea33c2aeae7c2200b"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "62df942c087c16d00fbcc9a6079c463b"
server_key_iv: "3789a0ecb2f222f17309cfa1f76f3d02"
server_key_version: "encryptionkey@0"
I20260809 07:52:30.004014 6733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "7f749dcc5135472ea33c2aeae7c2200b"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "62df942c087c16d00fbcc9a6079c463b"
server_key_iv: "3789a0ecb2f222f17309cfa1f76f3d02"
server_key_version: "encryptionkey@0"
I20260809 07:52:30.008064 6733 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:52:30.010630 6748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.011818 6733 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:52:30.011956 6733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "7f749dcc5135472ea33c2aeae7c2200b"
format_stamp: "Formatted at 2026-08-09 07:52:29 on dist-test-slave-10pc"
server_key: "62df942c087c16d00fbcc9a6079c463b"
server_key_iv: "3789a0ecb2f222f17309cfa1f76f3d02"
server_key_version: "encryptionkey@0"
I20260809 07:52:30.012073 6733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:30.027402 6733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:30.028098 6733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:30.028331 6733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:30.028978 6733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:30.029911 6733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:30.029985 6733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.030058 6733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:30.030102 6733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:30.039520 6733 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:34003
I20260809 07:52:30.039568 6873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:34003 every 8 connection(s)
I20260809 07:52:30.040418 6733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:30.044788 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 6733
I20260809 07:52:30.044960 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:52:30.045289 18717 external_mini_cluster.cc:1428] Setting key 48f5be0622563cfa2596e38c2db66c11
I20260809 07:52:30.050247 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.042249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:48599 (local address 127.18.71.126:35521)
0809 07:52:30.042439 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.042446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.043567 (+ 1121us) server_negotiation.cc:407] Connection header received
0809 07:52:30.045393 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.045399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.045460 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.045536 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:30.046673 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.047203 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.048423 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.048698 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.048780 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.049858 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:30.049861 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:30.049863 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:30.049873 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:30.049888 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:30.049895 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:30.049948 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.050093 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.050139 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:52:30.051124 6876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.042721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:35521 (local address 127.18.71.67:48599)
0809 07:52:30.043386 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:30.043427 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:30.045132 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:30.045755 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:30.045762 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:30.045847 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:30.046485 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.046507 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.047343 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.047346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.048308 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.048318 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.048839 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.048842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.048984 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.049618 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:30.049620 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:30.049663 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:30.049727 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:30.049969 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:30.049975 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:30.049980 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 07:52:30.050078 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 07:52:30.050321 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":115,"threads_started":1}
I20260809 07:52:30.052192 6874 heartbeater.cc:347] Connected to a master server at 127.18.71.126:35521
I20260809 07:52:30.052433 6874 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:30.052954 6874 heartbeater.cc:511] Master 127.18.71.126:35521 requested a full tablet report, sending...
I20260809 07:52:30.053921 6389 ts_manager.cc:194] Registered new tserver with Master: 7f749dcc5135472ea33c2aeae7c2200b (127.18.71.67:34003)
I20260809 07:52:30.054471 6389 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.67:48599
I20260809 07:52:30.059257 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:52:30.065032 6431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.061246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42430 (local address 127.18.71.126:35521)
0809 07:52:30.061427 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.061430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.061542 (+ 112us) server_negotiation.cc:407] Connection header received
0809 07:52:30.061641 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.061643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.061679 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.061753 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:30.062052 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.062535 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.063360 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.063557 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.063628 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.064518 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:30.064521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:30.064523 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:30.064535 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:30.064552 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:30.064559 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:30.064662 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.064800 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.064854 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 07:52:30.067984 6389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:52:30.071069 6389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:52:30.085795 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.079701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37884 (local address 127.18.71.66:37995)
0809 07:52:30.080183 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.080196 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.080225 (+ 29us) server_negotiation.cc:407] Connection header received
0809 07:52:30.080542 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.080549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.080767 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.080936 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:30.081432 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.082539 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.084385 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.085093 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.085216 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.085416 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.085517 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.085623 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":151,"threads_started":1}
I20260809 07:52:30.085795 6887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.079530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:37995 (local address 127.0.0.1:37884)
0809 07:52:30.080120 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:30.080170 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:30.080287 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:30.081118 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:30.081126 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:30.081149 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:30.081295 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.081302 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.082649 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.082652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.084215 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.084226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.085228 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.085232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.085338 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.085375 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 07:52:30.085489 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 07:52:30.085610 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":114,"thread_start_us":77,"threads_started":1}
I20260809 07:52:30.085870 6885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.079728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:43901 (local address 127.0.0.1:35808)
0809 07:52:30.080391 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:30.080405 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:30.080529 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:30.081261 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:30.081265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:30.081280 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:30.081366 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.081374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.082560 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.082566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.084323 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.084329 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.085437 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.085441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.085527 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.085550 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:52:30.085602 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 07:52:30.085684 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":227,"thread_start_us":116,"threads_started":1}
I20260809 07:52:30.086483 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.080286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:34003 (local address 127.0.0.1:50528)
0809 07:52:30.081610 (+ 1324us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:30.081624 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:30.081718 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:30.082147 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:30.082150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:30.082167 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:30.082245 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.082250 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.083294 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.083299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.084317 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.084329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.085675 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.085678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.085759 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.085780 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:52:30.085845 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 07:52:30.086357 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1264,"mutex_wait_us":6,"thread_start_us":92,"threads_started":1}
I20260809 07:52:30.086550 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.080357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.18.71.65:43901)
0809 07:52:30.080775 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.080788 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.080812 (+ 24us) server_negotiation.cc:407] Connection header received
0809 07:52:30.080890 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.080897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.081091 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.081290 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:30.081449 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.082456 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.084487 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.085293 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.086068 (+ 775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.086261 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.086311 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.086395 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":154,"threads_started":1}
I20260809 07:52:30.087067 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.080718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50528 (local address 127.18.71.67:34003)
0809 07:52:30.081177 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.081189 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.081746 (+ 557us) server_negotiation.cc:407] Connection header received
0809 07:52:30.081833 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.081838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.081999 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.082162 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:30.082349 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.083171 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.084488 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.085293 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.085412 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.085602 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.086816 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.086918 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":137,"threads_started":1}
I20260809 07:52:30.088972 6661 tablet_service.cc:1467] Processing CreateTablet for tablet 812791909ce44a3791c1c2aa79002201 (DEFAULT_TABLE table=test-table [id=1f06c986148b436d99203ca12455404a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:30.089398 6522 tablet_service.cc:1467] Processing CreateTablet for tablet 812791909ce44a3791c1c2aa79002201 (DEFAULT_TABLE table=test-table [id=1f06c986148b436d99203ca12455404a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:30.090075 6661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812791909ce44a3791c1c2aa79002201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:30.090404 6522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812791909ce44a3791c1c2aa79002201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:30.096374 6893 tablet_bootstrap.cc:492] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: Bootstrap starting.
I20260809 07:52:30.097126 6892 tablet_bootstrap.cc:492] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Bootstrap starting.
I20260809 07:52:30.098860 6893 tablet_bootstrap.cc:654] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:30.099301 6892 tablet_bootstrap.cc:654] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:30.098837 6802 tablet_service.cc:1467] Processing CreateTablet for tablet 812791909ce44a3791c1c2aa79002201 (DEFAULT_TABLE table=test-table [id=1f06c986148b436d99203ca12455404a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:30.099687 6893 log.cc:824] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:30.099992 6802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 812791909ce44a3791c1c2aa79002201. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:30.100103 6892 log.cc:824] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:30.102264 6892 tablet_bootstrap.cc:492] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: No bootstrap required, opened a new log
I20260809 07:52:30.102432 6892 ts_tablet_manager.cc:1397] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 07:52:30.104072 6893 tablet_bootstrap.cc:492] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: No bootstrap required, opened a new log
I20260809 07:52:30.104267 6893 ts_tablet_manager.cc:1397] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 07:52:30.107367 6896 tablet_bootstrap.cc:492] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Bootstrap starting.
I20260809 07:52:30.107993 6892 raft_consensus.cc:348] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.108255 6892 raft_consensus.cc:374] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:30.108311 6892 raft_consensus.cc:729] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c9fbd589634cb5b49b87730d9a58c7, State: Initialized, Role: FOLLOWER
I20260809 07:52:30.108781 6892 consensus_queue.cc:260] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.108757 6893 raft_consensus.cc:348] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.108989 6893 raft_consensus.cc:374] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:30.109045 6893 raft_consensus.cc:729] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 691d66335d3a4a748312da6682a10d0e, State: Initialized, Role: FOLLOWER
I20260809 07:52:30.109496 6893 consensus_queue.cc:260] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.109719 6896 tablet_bootstrap.cc:654] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:30.109854 6892 ts_tablet_manager.cc:1428] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 07:52:30.110205 6730 heartbeater.cc:503] Master 127.18.71.126:35521 was elected leader, sending a full tablet report...
I20260809 07:52:30.110495 6591 heartbeater.cc:503] Master 127.18.71.126:35521 was elected leader, sending a full tablet report...
I20260809 07:52:30.110565 6896 log.cc:824] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:30.111131 6893 ts_tablet_manager.cc:1428] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260809 07:52:30.112568 6896 tablet_bootstrap.cc:492] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: No bootstrap required, opened a new log
I20260809 07:52:30.112746 6896 ts_tablet_manager.cc:1397] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 07:52:30.116890 6896 raft_consensus.cc:348] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.117102 6896 raft_consensus.cc:374] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:30.117144 6896 raft_consensus.cc:729] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f749dcc5135472ea33c2aeae7c2200b, State: Initialized, Role: FOLLOWER
I20260809 07:52:30.117475 6896 consensus_queue.cc:260] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.118147 6874 heartbeater.cc:503] Master 127.18.71.126:35521 was elected leader, sending a full tablet report...
I20260809 07:52:30.118427 6896 ts_tablet_manager.cc:1428] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 07:52:30.163403 6592 tablet.cc:2369] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:52:30.165622 6902 raft_consensus.cc:482] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:30.165849 6902 raft_consensus.cc:504] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.167162 6902 leader_election.cc:288] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e5c9fbd589634cb5b49b87730d9a58c7 (127.18.71.66:37995), 691d66335d3a4a748312da6682a10d0e (127.18.71.65:43901)
I20260809 07:52:30.170847 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.167642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:59381 (local address 127.18.71.66:37995)
0809 07:52:30.167802 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.167806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.167818 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:52:30.167864 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.167866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.167906 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.167994 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:30.168280 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.168878 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.169908 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.170288 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.170413 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.170465 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.170655 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.170726 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 07:52:30.170847 6876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.167374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:37995 (local address 127.18.71.67:59381)
0809 07:52:30.167565 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:30.167582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:30.167685 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:30.168007 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:30.168010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:30.168029 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:30.168155 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.168161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.168992 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.168996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.169783 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.169792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.170413 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.170417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.170524 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.170549 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 07:52:30.170637 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 07:52:30.170711 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260809 07:52:30.171075 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.167545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:39857 (local address 127.18.71.65:43901)
0809 07:52:30.167679 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.167684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.167971 (+ 287us) server_negotiation.cc:407] Connection header received
0809 07:52:30.168138 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.168140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.168185 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.168271 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:30.168571 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.169054 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.170109 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.170515 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.170650 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.170699 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.170896 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.170967 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 07:52:30.171101 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.167374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:43901 (local address 127.18.71.67:39857)
0809 07:52:30.167859 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:30.167871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:30.168001 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:30.168283 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:30.168286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:30.168302 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:30.168426 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.168431 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.169176 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.169182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.169974 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:30.169983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.170663 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.170666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:30.170770 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.170787 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 07:52:30.170877 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 07:52:30.170945 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":39,"thread_start_us":133,"threads_started":1}
I20260809 07:52:30.171563 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812791909ce44a3791c1c2aa79002201" candidate_uuid: "7f749dcc5135472ea33c2aeae7c2200b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" is_pre_election: true
I20260809 07:52:30.171681 6543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812791909ce44a3791c1c2aa79002201" candidate_uuid: "7f749dcc5135472ea33c2aeae7c2200b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691d66335d3a4a748312da6682a10d0e" is_pre_election: true
I20260809 07:52:30.171918 6681 raft_consensus.cc:2454] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f749dcc5135472ea33c2aeae7c2200b in term 0.
I20260809 07:52:30.172005 6543 raft_consensus.cc:2454] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f749dcc5135472ea33c2aeae7c2200b in term 0.
I20260809 07:52:30.172443 6749 leader_election.cc:302] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f749dcc5135472ea33c2aeae7c2200b, e5c9fbd589634cb5b49b87730d9a58c7; no voters:
I20260809 07:52:30.172796 6902 raft_consensus.cc:2781] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:52:30.172890 6902 raft_consensus.cc:482] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:30.172935 6902 raft_consensus.cc:3037] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:30.173986 6902 raft_consensus.cc:504] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.174423 6902 leader_election.cc:288] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [CANDIDATE]: Term 1 election: Requested vote from peers e5c9fbd589634cb5b49b87730d9a58c7 (127.18.71.66:37995), 691d66335d3a4a748312da6682a10d0e (127.18.71.65:43901)
I20260809 07:52:30.174712 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812791909ce44a3791c1c2aa79002201" candidate_uuid: "7f749dcc5135472ea33c2aeae7c2200b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5c9fbd589634cb5b49b87730d9a58c7"
I20260809 07:52:30.174852 6681 raft_consensus.cc:3037] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:30.174870 6543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "812791909ce44a3791c1c2aa79002201" candidate_uuid: "7f749dcc5135472ea33c2aeae7c2200b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "691d66335d3a4a748312da6682a10d0e"
I20260809 07:52:30.174978 6543 raft_consensus.cc:3037] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:30.175985 6681 raft_consensus.cc:2454] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f749dcc5135472ea33c2aeae7c2200b in term 1.
I20260809 07:52:30.176100 6543 raft_consensus.cc:2454] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f749dcc5135472ea33c2aeae7c2200b in term 1.
I20260809 07:52:30.176368 6749 leader_election.cc:302] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f749dcc5135472ea33c2aeae7c2200b, e5c9fbd589634cb5b49b87730d9a58c7; no voters:
I20260809 07:52:30.176560 6902 raft_consensus.cc:2781] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:30.176818 6902 raft_consensus.cc:686] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 1 LEADER]: Becoming Leader. State: Replica: 7f749dcc5135472ea33c2aeae7c2200b, State: Running, Role: LEADER
I20260809 07:52:30.177201 6902 consensus_queue.cc:237] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } }
I20260809 07:52:30.180363 6388 catalog_manager.cc:5495] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f749dcc5135472ea33c2aeae7c2200b (127.18.71.67). New cstate: current_term: 1 leader_uuid: "7f749dcc5135472ea33c2aeae7c2200b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f749dcc5135472ea33c2aeae7c2200b" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 34003 } health_report { overall_health: HEALTHY } } }
I20260809 07:52:30.209865 6891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.206121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50542 (local address 127.18.71.67:34003)
0809 07:52:30.206257 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:30.206263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:30.206279 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:52:30.206389 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:30.206393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:30.206462 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:30.206582 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:52:30.206843 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.207355 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.208267 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:30.208528 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:30.208642 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:30.208960 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:52:30.209378 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:52:30.209540 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:30.209642 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 07:52:30.209679 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:52:30.215833 6681 raft_consensus.cc:1261] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 1 FOLLOWER]: Refusing update from remote peer 7f749dcc5135472ea33c2aeae7c2200b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:30.215858 6543 raft_consensus.cc:1261] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e [term 1 FOLLOWER]: Refusing update from remote peer 7f749dcc5135472ea33c2aeae7c2200b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:30.216635 6904 consensus_queue.cc:1035] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5c9fbd589634cb5b49b87730d9a58c7" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:30.216821 6902 consensus_queue.cc:1035] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [LEADER]: Connected to new peer: Peer: permanent_uuid: "691d66335d3a4a748312da6682a10d0e" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:30.224222 6910 mvcc.cc:205] Tried to move back new op lower bound from 7316528948076457984 to 7316528947928649728. Current Snapshot: MvccSnapshot[applied={T|T < 7316528948076457984}]
I20260809 07:52:30.225660 6909 mvcc.cc:205] Tried to move back new op lower bound from 7316528948076457984 to 7316528947928649728. Current Snapshot: MvccSnapshot[applied={T|T < 7316528948076457984}]
I20260809 07:52:30.232264 6389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:30.232409 6389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:30.234875 6389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47cc2ae1861a413aaf2fb481adca1727: Sending DeleteTablet for 3 replicas of tablet 812791909ce44a3791c1c2aa79002201
I20260809 07:52:30.235612 6522 tablet_service.cc:1514] Processing DeleteTablet for tablet 812791909ce44a3791c1c2aa79002201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:30 UTC) from {username='slave'} at 127.0.0.1:35808
I20260809 07:52:30.235661 6802 tablet_service.cc:1514] Processing DeleteTablet for tablet 812791909ce44a3791c1c2aa79002201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:30 UTC) from {username='slave'} at 127.0.0.1:50528
I20260809 07:52:30.236123 6919 tablet_replica.cc:330] T 812791909ce44a3791c1c2aa79002201 P 691d66335d3a4a748312da6682a10d0e: stopping tablet replica
I20260809 07:52:30.236129 6661 tablet_service.cc:1514] Processing DeleteTablet for tablet 812791909ce44a3791c1c2aa79002201 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:30 UTC) from {username='slave'} at 127.0.0.1:37884
I20260809 07:52:30.236423 6918 tablet_replica.cc:330] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: stopping tablet replica
I20260809 07:52:30.236620 6920 tablet_replica.cc:330] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: stopping tablet replica
I20260809 07:52:30.236919 6920 raft_consensus.cc:2229] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:30.237112 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 6454
I20260809 07:52:30.237272 6920 raft_consensus.cc:2258] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:30.241639 6918 raft_consensus.cc:2229] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:52:30.242120 6918 raft_consensus.cc:2258] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:30.243697 6918 ts_tablet_manager.cc:1905] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:30.246724 6918 ts_tablet_manager.cc:1918] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:30.246881 6918 log.cc:1196] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/812791909ce44a3791c1c2aa79002201
I20260809 07:52:30.247272 6918 ts_tablet_manager.cc:1939] T 812791909ce44a3791c1c2aa79002201 P 7f749dcc5135472ea33c2aeae7c2200b: Deleting consensus metadata
I20260809 07:52:30.248530 6372 catalog_manager.cc:4841] TS 7f749dcc5135472ea33c2aeae7c2200b (127.18.71.67:34003): tablet 812791909ce44a3791c1c2aa79002201 (table test-table [id=1f06c986148b436d99203ca12455404a]) successfully deleted
I20260809 07:52:30.254526 6920 ts_tablet_manager.cc:1905] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:30.258244 6920 ts_tablet_manager.cc:1918] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:30.258370 6920 log.cc:1196] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/812791909ce44a3791c1c2aa79002201
I20260809 07:52:30.258747 6920 ts_tablet_manager.cc:1939] T 812791909ce44a3791c1c2aa79002201 P e5c9fbd589634cb5b49b87730d9a58c7: Deleting consensus metadata
I20260809 07:52:30.260227 6372 catalog_manager.cc:4841] TS e5c9fbd589634cb5b49b87730d9a58c7 (127.18.71.66:37995): tablet 812791909ce44a3791c1c2aa79002201 (table test-table [id=1f06c986148b436d99203ca12455404a]) successfully deleted
W20260809 07:52:30.261440 6376 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:43901 (error 108)
I20260809 07:52:30.262508 6889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:30.262182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:43901 (local address 127.0.0.1:35814)
0809 07:52:30.262324 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:30.262404 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:43901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 07:52:30.262638 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 6594
W20260809 07:52:30.265686 6376 catalog_manager.cc:4584] TS 691d66335d3a4a748312da6682a10d0e (127.18.71.65:43901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 812791909ce44a3791c1c2aa79002201: Network error: Client connection negotiation failed: client connection to 127.18.71.65:43901: connect: Connection refused (error 111)
I20260809 07:52:30.271126 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 6733
I20260809 07:52:30.280272 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 6357
2026-08-09T07:52:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1075 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:30 dist-test-slave-10pc krb5kdc[6926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:30 dist-test-slave-10pc krb5kdc[6926](info): set up 2 sockets
Aug 09 07:52:30 dist-test-slave-10pc krb5kdc[6926](info): commencing operation
krb5kdc: starting...
W20260809 07:52:31.355731 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:31 dist-test-slave-10pc krb5kdc[6926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:31.483465 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33487
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:33487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:31.595539 6943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:31.595836 6943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:31.595917 6943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:31.599941 6943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:31.600026 6943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:31.600065 6943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:31.600128 6943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:31.600180 6943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:31.605525 6943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:33487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:33487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:31.606616 6943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:31.607614 6943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:31.616292 6951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:31.616302 6949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:31.616309 6948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:31.616518 6943 server_base.cc:1034] running on GCE node
I20260809 07:52:31.617190 6943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:31.618233 6943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:31.619421 6943 hybrid_clock.cc:648] HybridClock initialized: now 1786261951619396 us; error 33 us; skew 500 ppm
Aug 09 07:52:31 dist-test-slave-10pc krb5kdc[6926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261951, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:31.622233 6943 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:31.623042 6943 webserver.cc:460] Webserver started at http://127.18.71.126:42717/ using document root <none> and password file <none>
I20260809 07:52:31.623543 6943 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:31.623601 6943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:31.623759 6943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:31.625783 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "1eaab99fd13f4d09ad2b60a286fe4e57"
format_stamp: "Formatted at 2026-08-09 07:52:31 on dist-test-slave-10pc"
server_key: "69b2fa34c4efd3e17d1719d9670ca523"
server_key_iv: "f4989ba0d3bc48be693f05983f0d2961"
server_key_version: "encryptionkey@0"
I20260809 07:52:31.626297 6943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "1eaab99fd13f4d09ad2b60a286fe4e57"
format_stamp: "Formatted at 2026-08-09 07:52:31 on dist-test-slave-10pc"
server_key: "69b2fa34c4efd3e17d1719d9670ca523"
server_key_iv: "f4989ba0d3bc48be693f05983f0d2961"
server_key_version: "encryptionkey@0"
I20260809 07:52:31.630064 6943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:52:31.632381 6958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:31.633652 6943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:31.633754 6943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "1eaab99fd13f4d09ad2b60a286fe4e57"
format_stamp: "Formatted at 2026-08-09 07:52:31 on dist-test-slave-10pc"
server_key: "69b2fa34c4efd3e17d1719d9670ca523"
server_key_iv: "f4989ba0d3bc48be693f05983f0d2961"
server_key_version: "encryptionkey@0"
I20260809 07:52:31.633839 6943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:31.667341 6943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:31.670475 6943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:31.670647 6943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:31.677757 6943 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:33487
I20260809 07:52:31.677785 7013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:33487 every 8 connection(s)
I20260809 07:52:31.678726 6943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:31.680400 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 6943
I20260809 07:52:31.680584 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:31.680910 18717 external_mini_cluster.cc:1428] Setting key 4398d01eeec5f9cb573d33f34d268f09
I20260809 07:52:31.681829 7014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:31.688634 7014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57: Bootstrap starting.
Aug 09 07:52:31 dist-test-slave-10pc krb5kdc[6926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:31.690918 7014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:31.691800 7014 log.cc:824] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:31.693813 7014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57: No bootstrap required, opened a new log
I20260809 07:52:31.696305 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:31.682347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.18.71.126:33487)
0809 07:52:31.682820 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:31.682839 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:31.682882 (+ 43us) server_negotiation.cc:407] Connection header received
0809 07:52:31.683736 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:31.683761 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:31.684171 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:31.684572 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:31.685264 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:31.686808 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:31.687752 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:31.688413 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:31.688592 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:31.691116 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:31.691121 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:31.691135 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:31.691161 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:31.693127 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:31.693598 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:31.693603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:31.693607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:31.693715 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:31.694033 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:31.694037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:31.694038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:31.694556 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:31.694778 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:31.694990 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 07:52:31.695285 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260809 07:52:31.697940 7014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33487 } }
I20260809 07:52:31.698189 7014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:31.698251 7014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eaab99fd13f4d09ad2b60a286fe4e57, State: Initialized, Role: FOLLOWER
I20260809 07:52:31.698649 7014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33487 } }
I20260809 07:52:31.698817 7014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:31.698904 7014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:31.699021 7014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:31.699934 7014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33487 } }
I20260809 07:52:31.700286 7014 leader_election.cc:302] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1eaab99fd13f4d09ad2b60a286fe4e57; no voters:
I20260809 07:52:31.700645 7014 leader_election.cc:288] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:31.700763 7019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:31.701015 7019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [term 1 LEADER]: Becoming Leader. State: Replica: 1eaab99fd13f4d09ad2b60a286fe4e57, State: Running, Role: LEADER
I20260809 07:52:31.701380 7019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33487 } }
I20260809 07:52:31.701929 7014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:31.703068 7020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33487 } } }
I20260809 07:52:31.703197 7020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:31.703428 7022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1eaab99fd13f4d09ad2b60a286fe4e57. Latest consensus state: current_term: 1 leader_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1eaab99fd13f4d09ad2b60a286fe4e57" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 33487 } } }
I20260809 07:52:31.703533 7022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:31.703814 7030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:31.706911 7030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:31.712013 7030 catalog_manager.cc:1347] Generated new cluster ID: 1603b855ad884524b28244b962658fcf
I20260809 07:52:31.712092 7030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:31.722656 7030 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:31.723554 7030 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:31.730289 7030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57: Generated new TSK 0
I20260809 07:52:31.730957 7030 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.18.71.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
Entry for principal kudu/127.18.71.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab.
I20260809 07:52:31.767475 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33487
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:31.891639 7043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:31.891884 7043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:31.891927 7043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:31.896039 7043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:31.896095 7043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:31.896150 7043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:52:31.901038 7043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.65.keytab
--principal=kudu/127.18.71.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:31.902055 7043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:31.903152 7043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:31.912796 7050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:31.912796 7048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:31.912793 7052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:31.913050 7043 server_base.cc:1034] running on GCE node
I20260809 07:52:31.913452 7043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:31.914126 7043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:31.915336 7043 hybrid_clock.cc:648] HybridClock initialized: now 1786261951915317 us; error 29 us; skew 500 ppm
Aug 09 07:52:31 dist-test-slave-10pc krb5kdc[6926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261951, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:31.918104 7043 init.cc:377] Logged in from keytab as kudu/127.18.71.65@KRBTEST.COM (short username kudu)
I20260809 07:52:31.919003 7043 webserver.cc:460] Webserver started at http://127.18.71.65:39271/ using document root <none> and password file <none>
I20260809 07:52:31.919579 7043 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:31.919641 7043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:31.919874 7043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:31.921968 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "810808e3ab3a4da0913034d195233d63"
format_stamp: "Formatted at 2026-08-09 07:52:31 on dist-test-slave-10pc"
server_key: "88b5d9579cd50143566e34caccc7db03"
server_key_iv: "eab1c8d8e63aecf9c6ce978bab464c4c"
server_key_version: "encryptionkey@0"
I20260809 07:52:31.922582 7043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "810808e3ab3a4da0913034d195233d63"
format_stamp: "Formatted at 2026-08-09 07:52:31 on dist-test-slave-10pc"
server_key: "88b5d9579cd50143566e34caccc7db03"
server_key_iv: "eab1c8d8e63aecf9c6ce978bab464c4c"
server_key_version: "encryptionkey@0"
I20260809 07:52:31.926400 7043 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:31.928889 7060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:31.930101 7043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:31.930234 7043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "810808e3ab3a4da0913034d195233d63"
format_stamp: "Formatted at 2026-08-09 07:52:31 on dist-test-slave-10pc"
server_key: "88b5d9579cd50143566e34caccc7db03"
server_key_iv: "eab1c8d8e63aecf9c6ce978bab464c4c"
server_key_version: "encryptionkey@0"
I20260809 07:52:31.930343 7043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:31.952809 7043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:31.955782 7043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:31.956034 7043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:31.956724 7043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:31.957564 7043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:31.957648 7043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:31.957738 7043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:31.957790 7043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:31.966593 7043 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:40009
I20260809 07:52:31.966621 7178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:40009 every 8 connection(s)
I20260809 07:52:31.967476 7043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:31.975296 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 7043
I20260809 07:52:31.975451 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:52:31.975764 18717 external_mini_cluster.cc:1428] Setting key a29ff37db6ff2b697c441ee0e6edf129
Aug 09 07:52:31 dist-test-slave-10pc krb5kdc[6926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261951, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.65@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:31.981983 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:31.969324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:41427 (local address 127.18.71.126:33487)
0809 07:52:31.969459 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:31.969462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:31.970380 (+ 918us) server_negotiation.cc:407] Connection header received
0809 07:52:31.971755 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:31.971759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:31.971816 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:31.971909 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:31.973173 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:31.973730 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:31.974886 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:31.975158 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:31.975272 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:31.978123 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:31.978126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:31.978128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:31.978151 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:31.979775 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:31.980270 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:31.980275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:31.980277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:31.980374 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:31.980753 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:31.980757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:31.980759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:31.981007 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:31.981114 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:31.981591 (+ 477us) server_negotiation.cc:299] Negotiation successful
0809 07:52:31.981740 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:52:31.982596 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:31.969699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33487 (local address 127.18.71.65:41427)
0809 07:52:31.970216 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:31.970280 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:31.971544 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:31.972080 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:31.972087 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:31.972438 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:31.973014 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:31.973036 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:31.973865 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:31.973874 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:31.974750 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:31.974759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:31.975314 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:31.975317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:31.975554 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:31.976214 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:31.976232 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:31.977955 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:31.979897 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:31.979902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:31.979913 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:31.980168 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:31.980509 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:31.980516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:31.980517 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:31.980657 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:31.981113 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:31.981120 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:31.981338 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 07:52:31.981482 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 07:52:31.981733 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":111,"threads_started":1}
I20260809 07:52:31.983963 7179 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33487
I20260809 07:52:31.984234 7179 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:31.984805 7179 heartbeater.cc:511] Master 127.18.71.126:33487 requested a full tablet report, sending...
I20260809 07:52:31.986568 6975 ts_manager.cc:194] Registered new tserver with Master: 810808e3ab3a4da0913034d195233d63 (127.18.71.65:40009)
WARNING: no policy specified for kudu/127.18.71.66@KRBTEST.COM; defaulting to no policy
I20260809 07:52:31.988180 6975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.65@KRBTEST.COM'} at 127.18.71.65:41427
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
Entry for principal kudu/127.18.71.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab.
I20260809 07:52:32.007323 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33487
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:32.120450 7184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:32.120764 7184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:32.120805 7184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:32.124799 7184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:32.124846 7184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:32.124902 7184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:52:32.129940 7184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.66.keytab
--principal=kudu/127.18.71.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:32.130863 7184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:32.131807 7184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:32.141999 7191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:32.141996 7193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:32.141996 7190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:32.142295 7184 server_base.cc:1034] running on GCE node
I20260809 07:52:32.142725 7184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:32.143319 7184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:32.144541 7184 hybrid_clock.cc:648] HybridClock initialized: now 1786261952144526 us; error 39 us; skew 500 ppm
Aug 09 07:52:32 dist-test-slave-10pc krb5kdc[6926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261952, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:32.147389 7184 init.cc:377] Logged in from keytab as kudu/127.18.71.66@KRBTEST.COM (short username kudu)
I20260809 07:52:32.148321 7184 webserver.cc:460] Webserver started at http://127.18.71.66:36549/ using document root <none> and password file <none>
I20260809 07:52:32.148926 7184 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:32.148989 7184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:32.149260 7184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:32.151297 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "d5378d228cb14719ba0b27115aabe9c4"
format_stamp: "Formatted at 2026-08-09 07:52:32 on dist-test-slave-10pc"
server_key: "b0fbb9687e9096ad39f87eb9ba9b0243"
server_key_iv: "c2e3c444a775b07857622440e0d0eabd"
server_key_version: "encryptionkey@0"
I20260809 07:52:32.151852 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "d5378d228cb14719ba0b27115aabe9c4"
format_stamp: "Formatted at 2026-08-09 07:52:32 on dist-test-slave-10pc"
server_key: "b0fbb9687e9096ad39f87eb9ba9b0243"
server_key_iv: "c2e3c444a775b07857622440e0d0eabd"
server_key_version: "encryptionkey@0"
I20260809 07:52:32.155534 7184 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:52:32.157979 7201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:32.159128 7184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:52:32.159248 7184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "d5378d228cb14719ba0b27115aabe9c4"
format_stamp: "Formatted at 2026-08-09 07:52:32 on dist-test-slave-10pc"
server_key: "b0fbb9687e9096ad39f87eb9ba9b0243"
server_key_iv: "c2e3c444a775b07857622440e0d0eabd"
server_key_version: "encryptionkey@0"
I20260809 07:52:32.159353 7184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:32.194782 7184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:32.197899 7184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:32.198122 7184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:32.198752 7184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:32.199609 7184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:32.199657 7184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:32.199728 7184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:32.199766 7184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:32.209537 7184 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:33631
I20260809 07:52:32.209563 7322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:33631 every 8 connection(s)
I20260809 07:52:32.210438 7184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:32.214536 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 7184
I20260809 07:52:32.214640 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:52:32.214919 18717 external_mini_cluster.cc:1428] Setting key 9ad1934254babc8713d2549390b12869
Aug 09 07:52:32 dist-test-slave-10pc krb5kdc[6926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261952, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.66@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.18.71.67@KRBTEST.COM; defaulting to no policy
I20260809 07:52:32.225769 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.212604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:37293 (local address 127.18.71.126:33487)
0809 07:52:32.212744 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.212748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.213707 (+ 959us) server_negotiation.cc:407] Connection header received
0809 07:52:32.215197 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.215201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.215256 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.215358 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:32.216757 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.217560 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.218799 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.219176 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.219299 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.221988 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:32.221991 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:32.221993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:32.222016 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:32.223584 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:32.224136 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:32.224141 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:32.224143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:32.224228 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:32.224639 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:32.224643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:32.224644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:32.224825 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:32.224934 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.225454 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.225581 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:52:32.226554 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.212992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33487 (local address 127.18.71.66:37293)
0809 07:52:32.213559 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.213603 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:32.214946 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:32.215565 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:32.215578 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:32.215935 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:32.216606 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.216624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.217694 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.217696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.218677 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.218687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.219329 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.219332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.219468 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.220102 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:32.220127 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:32.221814 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:32.223721 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:32.223727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:32.223762 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:32.224025 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:32.224323 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:32.224326 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:32.224328 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:32.224522 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:32.224936 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:32.224955 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:32.225210 (+ 255us) client_negotiation.cc:769] Sending connection context
0809 07:52:32.225392 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 07:52:32.225716 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":191,"threads_started":1}
I20260809 07:52:32.227700 7323 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33487
I20260809 07:52:32.228008 7323 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.67@KRBTEST.COM" created.
I20260809 07:52:32.228590 7323 heartbeater.cc:511] Master 127.18.71.126:33487 requested a full tablet report, sending...
I20260809 07:52:32.229753 6975 ts_manager.cc:194] Registered new tserver with Master: d5378d228cb14719ba0b27115aabe9c4 (127.18.71.66:33631)
I20260809 07:52:32.230648 6975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.66@KRBTEST.COM'} at 127.18.71.66:37293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
Entry for principal kudu/127.18.71.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab.
I20260809 07:52:32.245838 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33487
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:32.360565 7328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:32.360852 7328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:32.360922 7328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:32.365010 7328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:32.365096 7328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:32.365185 7328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:52:32.370507 7328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.67.keytab
--principal=kudu/127.18.71.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:33487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:32.371719 7328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:32.372836 7328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:32.382495 7336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:32.382599 7333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:32.382599 7334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:32.382647 7328 server_base.cc:1034] running on GCE node
I20260809 07:52:32.383275 7328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:32.383915 7328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:32.385097 7328 hybrid_clock.cc:648] HybridClock initialized: now 1786261952385062 us; error 46 us; skew 500 ppm
Aug 09 07:52:32 dist-test-slave-10pc krb5kdc[6926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261952, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:32.387903 7328 init.cc:377] Logged in from keytab as kudu/127.18.71.67@KRBTEST.COM (short username kudu)
I20260809 07:52:32.388860 7328 webserver.cc:460] Webserver started at http://127.18.71.67:38395/ using document root <none> and password file <none>
I20260809 07:52:32.389425 7328 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:32.389516 7328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:32.389760 7328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:32.391788 7328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "add76052835c480b9f07bc0fe7cfc209"
format_stamp: "Formatted at 2026-08-09 07:52:32 on dist-test-slave-10pc"
server_key: "ef421b777be8e986b3641261f747a3a5"
server_key_iv: "f4a50046464c4ae405f924998607a9b6"
server_key_version: "encryptionkey@0"
I20260809 07:52:32.392355 7328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "add76052835c480b9f07bc0fe7cfc209"
format_stamp: "Formatted at 2026-08-09 07:52:32 on dist-test-slave-10pc"
server_key: "ef421b777be8e986b3641261f747a3a5"
server_key_iv: "f4a50046464c4ae405f924998607a9b6"
server_key_version: "encryptionkey@0"
I20260809 07:52:32.395987 7328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 07:52:32.398396 7344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:32.399521 7328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:32.399662 7328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "add76052835c480b9f07bc0fe7cfc209"
format_stamp: "Formatted at 2026-08-09 07:52:32 on dist-test-slave-10pc"
server_key: "ef421b777be8e986b3641261f747a3a5"
server_key_iv: "f4a50046464c4ae405f924998607a9b6"
server_key_version: "encryptionkey@0"
I20260809 07:52:32.399770 7328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:32.412277 7328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:32.415721 7328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:32.415925 7328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:32.416579 7328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:32.417414 7328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:32.417496 7328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:32.417593 7328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:32.417645 7328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:32.427556 7328 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:35361
I20260809 07:52:32.427587 7468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:35361 every 8 connection(s)
I20260809 07:52:32.428568 7328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:32.432338 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 7328
I20260809 07:52:32.432504 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:52:32.432893 18717 external_mini_cluster.cc:1428] Setting key c568315d51c2c3ac994e384bdd6d898f
Aug 09 07:52:32 dist-test-slave-10pc krb5kdc[6926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261952, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.67@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:32.443109 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.430466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:59381 (local address 127.18.71.126:33487)
0809 07:52:32.430638 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.430643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.431450 (+ 807us) server_negotiation.cc:407] Connection header received
0809 07:52:32.433041 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.433044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.433115 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.433219 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:32.434748 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.435471 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.436719 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.436930 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.437053 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.439758 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:32.439762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:32.439764 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:32.439788 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:32.441359 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:32.441826 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:32.441829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:32.441830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:32.441902 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:32.442197 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:32.442199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:32.442200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:32.442362 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:32.442436 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.442895 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.442990 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 07:52:32.443912 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.430851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:33487 (local address 127.18.71.67:59381)
0809 07:52:32.431302 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.431343 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:32.432805 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:32.433403 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:32.433410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:32.433973 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:32.434600 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.434621 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.435590 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.435593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.436599 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.436613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.437208 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.437220 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.437366 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.437932 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:52:32.437948 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:32.439575 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:32.441462 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:32.441467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:32.441492 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:32.441738 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:32.441983 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:52:32.441985 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:52:32.441986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:52:32.442118 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:52:32.442448 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:32.442452 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:32.442693 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 07:52:32.442850 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 07:52:32.443106 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":121,"threads_started":1}
I20260809 07:52:32.445115 7469 heartbeater.cc:347] Connected to a master server at 127.18.71.126:33487
I20260809 07:52:32.445369 7469 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:32.445916 7469 heartbeater.cc:511] Master 127.18.71.126:33487 requested a full tablet report, sending...
I20260809 07:52:32.447100 6975 ts_manager.cc:194] Registered new tserver with Master: add76052835c480b9f07bc0fe7cfc209 (127.18.71.67:35361)
I20260809 07:52:32.447669 6975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.18.71.67@KRBTEST.COM'} at 127.18.71.67:59381
I20260809 07:52:32.458784 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:52:32.468236 7017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.460760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51070 (local address 127.18.71.126:33487)
0809 07:52:32.460917 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.460921 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.461115 (+ 194us) server_negotiation.cc:407] Connection header received
0809 07:52:32.461237 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.461240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.461291 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.461379 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:32.462031 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.462537 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.463398 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.463626 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.463705 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.465197 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:32.465200 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:32.465202 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:32.465226 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:32.466681 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:32.467097 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:32.467099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:32.467101 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:32.467182 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:32.467410 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:32.467412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:32.467413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:32.467620 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:32.467724 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.467918 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.468014 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 07:52:32.471208 6975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:52:32.474398 6975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:52:32.489161 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.482950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.18.71.67:35361)
0809 07:52:32.483566 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.483579 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.483604 (+ 25us) server_negotiation.cc:407] Connection header received
0809 07:52:32.483703 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.483709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.483938 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.484487 (+ 549us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:32.484674 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.485890 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.487619 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.488397 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.488679 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.488835 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.488890 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.488989 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":172,"threads_started":1}
I20260809 07:52:32.489174 7482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.483549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40009 (local address 127.0.0.1:40016)
0809 07:52:32.483816 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.483830 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:32.483914 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:32.484651 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:32.484656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:32.484673 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:32.484786 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.484793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.486240 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.486244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.487433 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.487444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.488579 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.488583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.488704 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.488750 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 07:52:32.488844 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 07:52:32.489001 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":90,"thread_start_us":117,"threads_started":1}
I20260809 07:52:32.489429 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.483732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.18.71.65:40009)
0809 07:52:32.484178 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.484185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.484206 (+ 21us) server_negotiation.cc:407] Connection header received
0809 07:52:32.484278 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.484284 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.484500 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.484936 (+ 436us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:32.485082 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.486134 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.487564 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.488397 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.488967 (+ 570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.489113 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.489167 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.489267 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":71,"threads_started":1}
I20260809 07:52:32.489176 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.482738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:35361 (local address 127.0.0.1:48236)
0809 07:52:32.483089 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.483134 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:32.483293 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:32.484126 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:32.484132 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:32.484155 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:32.484305 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.484311 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.486035 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.486039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.487452 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.487461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.488579 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.488583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.488729 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.488752 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 07:52:32.488843 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 07:52:32.489001 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":107,"thread_start_us":108,"threads_started":1}
I20260809 07:52:32.489538 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.483898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:33631 (local address 127.0.0.1:56892)
0809 07:52:32.484786 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.484797 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:32.484915 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:32.485342 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:32.485345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:32.485363 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:32.485470 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.485479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.486669 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.486672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.487709 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.487716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.488506 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.488510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.489221 (+ 711us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.489245 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:52:32.489322 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 07:52:32.489393 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"mutex_wait_us":28,"thread_start_us":108,"threads_started":1}
I20260809 07:52:32.489948 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.484238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.18.71.66:33631)
0809 07:52:32.484714 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.484731 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.484886 (+ 155us) server_negotiation.cc:407] Connection header received
0809 07:52:32.485077 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.485081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.485212 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.485364 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:32.485573 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.486523 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.487818 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.488364 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.488525 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.488701 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.489680 (+ 979us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.489789 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":132,"threads_started":1}
I20260809 07:52:32.492786 7110 tablet_service.cc:1467] Processing CreateTablet for tablet 58ccfefc33be4d0ab7d00843293ccb29 (DEFAULT_TABLE table=test-table [id=368ef102cc574470af6614e3817f3263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:32.492791 7252 tablet_service.cc:1467] Processing CreateTablet for tablet 58ccfefc33be4d0ab7d00843293ccb29 (DEFAULT_TABLE table=test-table [id=368ef102cc574470af6614e3817f3263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:32.493963 7110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ccfefc33be4d0ab7d00843293ccb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:32.493963 7252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ccfefc33be4d0ab7d00843293ccb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:32.494252 7396 tablet_service.cc:1467] Processing CreateTablet for tablet 58ccfefc33be4d0ab7d00843293ccb29 (DEFAULT_TABLE table=test-table [id=368ef102cc574470af6614e3817f3263]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:32.495292 7396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58ccfefc33be4d0ab7d00843293ccb29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:32.501065 7486 tablet_bootstrap.cc:492] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Bootstrap starting.
I20260809 07:52:32.502022 7487 tablet_bootstrap.cc:492] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: Bootstrap starting.
I20260809 07:52:32.502022 7488 tablet_bootstrap.cc:492] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Bootstrap starting.
I20260809 07:52:32.503628 7486 tablet_bootstrap.cc:654] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:32.504273 7487 tablet_bootstrap.cc:654] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:32.504761 7488 tablet_bootstrap.cc:654] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:32.505157 7486 log.cc:824] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:32.505364 7487 log.cc:824] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:32.505635 7488 log.cc:824] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:32.507299 7488 tablet_bootstrap.cc:492] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: No bootstrap required, opened a new log
I20260809 07:52:32.507330 7487 tablet_bootstrap.cc:492] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: No bootstrap required, opened a new log
I20260809 07:52:32.507330 7486 tablet_bootstrap.cc:492] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: No bootstrap required, opened a new log
I20260809 07:52:32.507510 7487 ts_tablet_manager.cc:1397] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:52:32.507509 7486 ts_tablet_manager.cc:1397] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 07:52:32.507509 7488 ts_tablet_manager.cc:1397] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 07:52:32.510334 7488 raft_consensus.cc:348] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.510586 7488 raft_consensus.cc:374] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:32.510669 7488 raft_consensus.cc:729] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5378d228cb14719ba0b27115aabe9c4, State: Initialized, Role: FOLLOWER
I20260809 07:52:32.511322 7488 consensus_queue.cc:260] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.511754 7486 raft_consensus.cc:348] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.511838 7487 raft_consensus.cc:348] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.512042 7487 raft_consensus.cc:374] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:32.512106 7486 raft_consensus.cc:374] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:32.512151 7487 raft_consensus.cc:729] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 810808e3ab3a4da0913034d195233d63, State: Initialized, Role: FOLLOWER
I20260809 07:52:32.512362 7323 heartbeater.cc:503] Master 127.18.71.126:33487 was elected leader, sending a full tablet report...
I20260809 07:52:32.512390 7488 ts_tablet_manager.cc:1428] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 07:52:32.512643 7487 consensus_queue.cc:260] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.512845 7486 raft_consensus.cc:729] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: add76052835c480b9f07bc0fe7cfc209, State: Initialized, Role: FOLLOWER
I20260809 07:52:32.513278 7486 consensus_queue.cc:260] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.513628 7487 ts_tablet_manager.cc:1428] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 07:52:32.513962 7179 heartbeater.cc:503] Master 127.18.71.126:33487 was elected leader, sending a full tablet report...
I20260809 07:52:32.513979 7469 heartbeater.cc:503] Master 127.18.71.126:33487 was elected leader, sending a full tablet report...
I20260809 07:52:32.514225 7486 ts_tablet_manager.cc:1428] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260809 07:52:32.679548 7470 tablet.cc:2369] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:52:32.711601 7324 tablet.cc:2369] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 07:52:32.718936 7180 tablet.cc:2369] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:52:32.786074 7492 raft_consensus.cc:482] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:32.786291 7492 raft_consensus.cc:504] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.787398 7492 leader_election.cc:288] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers add76052835c480b9f07bc0fe7cfc209 (127.18.71.67:35361), 810808e3ab3a4da0913034d195233d63 (127.18.71.65:40009)
I20260809 07:52:32.791685 7481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.787818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:51651 (local address 127.18.71.67:35361)
0809 07:52:32.787950 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.787954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.788172 (+ 218us) server_negotiation.cc:407] Connection header received
0809 07:52:32.788241 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.788244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.788297 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.788399 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:32.788842 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.789461 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.790633 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.791087 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.791188 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.791227 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.791469 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.791543 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:52:32.791706 7495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.787680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:35361 (local address 127.18.71.66:51651)
0809 07:52:32.787948 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.787967 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:32.788092 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:32.788428 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:32.788433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:32.788490 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:32.788699 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.788706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.789567 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.789570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.790501 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.790510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.791221 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.791225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.791333 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.791363 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 07:52:32.791436 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 07:52:32.791534 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"spinlock_wait_cycles":34560,"thread_start_us":105,"threads_started":1}
I20260809 07:52:32.791834 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.788240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:47991 (local address 127.18.71.65:40009)
0809 07:52:32.788418 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.788422 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.788541 (+ 119us) server_negotiation.cc:407] Connection header received
0809 07:52:32.788708 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.788712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.788781 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.788858 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:32.789133 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.789753 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.790780 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.791264 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.791376 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.791441 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.791634 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.791707 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 07:52:32.792083 7496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.788080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40009 (local address 127.18.71.66:47991)
0809 07:52:32.788429 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.788443 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:32.788589 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:32.788886 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:32.788889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:32.788907 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:32.789022 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.789028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.789866 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.789869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.790662 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:32.790672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.791383 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.791386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:32.791506 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.791528 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 07:52:32.791604 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 07:52:32.791678 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"spinlock_wait_cycles":3072,"thread_start_us":93,"threads_started":1}
I20260809 07:52:32.792371 7419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ccfefc33be4d0ab7d00843293ccb29" candidate_uuid: "d5378d228cb14719ba0b27115aabe9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add76052835c480b9f07bc0fe7cfc209" is_pre_election: true
I20260809 07:52:32.792730 7419 raft_consensus.cc:2454] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5378d228cb14719ba0b27115aabe9c4 in term 0.
I20260809 07:52:32.792725 7130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ccfefc33be4d0ab7d00843293ccb29" candidate_uuid: "d5378d228cb14719ba0b27115aabe9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "810808e3ab3a4da0913034d195233d63" is_pre_election: true
I20260809 07:52:32.793004 7130 raft_consensus.cc:2454] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5378d228cb14719ba0b27115aabe9c4 in term 0.
I20260809 07:52:32.793251 7205 leader_election.cc:302] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: add76052835c480b9f07bc0fe7cfc209, d5378d228cb14719ba0b27115aabe9c4; no voters:
I20260809 07:52:32.793522 7492 raft_consensus.cc:2781] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:52:32.793601 7492 raft_consensus.cc:482] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:32.793632 7492 raft_consensus.cc:3037] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:32.794687 7492 raft_consensus.cc:504] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.795027 7492 leader_election.cc:288] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [CANDIDATE]: Term 1 election: Requested vote from peers add76052835c480b9f07bc0fe7cfc209 (127.18.71.67:35361), 810808e3ab3a4da0913034d195233d63 (127.18.71.65:40009)
I20260809 07:52:32.795428 7419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ccfefc33be4d0ab7d00843293ccb29" candidate_uuid: "d5378d228cb14719ba0b27115aabe9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "add76052835c480b9f07bc0fe7cfc209"
I20260809 07:52:32.795473 7130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58ccfefc33be4d0ab7d00843293ccb29" candidate_uuid: "d5378d228cb14719ba0b27115aabe9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "810808e3ab3a4da0913034d195233d63"
I20260809 07:52:32.795562 7419 raft_consensus.cc:3037] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:32.795572 7130 raft_consensus.cc:3037] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:32.796895 7130 raft_consensus.cc:2454] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5378d228cb14719ba0b27115aabe9c4 in term 1.
I20260809 07:52:32.796895 7419 raft_consensus.cc:2454] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5378d228cb14719ba0b27115aabe9c4 in term 1.
I20260809 07:52:32.797199 7205 leader_election.cc:302] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 810808e3ab3a4da0913034d195233d63, d5378d228cb14719ba0b27115aabe9c4; no voters:
I20260809 07:52:32.797387 7492 raft_consensus.cc:2781] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:32.797632 7492 raft_consensus.cc:686] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 1 LEADER]: Becoming Leader. State: Replica: d5378d228cb14719ba0b27115aabe9c4, State: Running, Role: LEADER
I20260809 07:52:32.797955 7492 consensus_queue.cc:237] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } }
I20260809 07:52:32.801013 6975 catalog_manager.cc:5495] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5378d228cb14719ba0b27115aabe9c4 (127.18.71.66). New cstate: current_term: 1 leader_uuid: "d5378d228cb14719ba0b27115aabe9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5378d228cb14719ba0b27115aabe9c4" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 33631 } health_report { overall_health: HEALTHY } } }
I20260809 07:52:32.849881 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.846672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56908 (local address 127.18.71.66:33631)
0809 07:52:32.846802 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:32.846806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:32.846818 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:52:32.846898 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:32.846900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:32.846956 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:32.847052 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:52:32.847280 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.847788 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.848612 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:32.848817 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:32.848890 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:32.849161 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:52:32.849573 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:52:32.849662 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:32.849724 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 07:52:32.849774 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:52:32.855762 7130 raft_consensus.cc:1261] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 1 FOLLOWER]: Refusing update from remote peer d5378d228cb14719ba0b27115aabe9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:32.855762 7419 raft_consensus.cc:1261] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 1 FOLLOWER]: Refusing update from remote peer d5378d228cb14719ba0b27115aabe9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:32.856525 7497 consensus_queue.cc:1035] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "810808e3ab3a4da0913034d195233d63" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 40009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:32.856705 7492 consensus_queue.cc:1035] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "add76052835c480b9f07bc0fe7cfc209" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 35361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:32.864091 7500 mvcc.cc:205] Tried to move back new op lower bound from 7316528958889893888 to 7316528958662885376. Current Snapshot: MvccSnapshot[applied={T|T < 7316528958889893888}]
I20260809 07:52:32.864923 7502 mvcc.cc:205] Tried to move back new op lower bound from 7316528958889893888 to 7316528958662885376. Current Snapshot: MvccSnapshot[applied={T|T < 7316528958889893888}]
I20260809 07:52:32.867429 7501 mvcc.cc:205] Tried to move back new op lower bound from 7316528958889893888 to 7316528958662885376. Current Snapshot: MvccSnapshot[applied={T|T < 7316528958889893888}]
I20260809 07:52:32.875399 6974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:32.875597 6974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:32.877892 6974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1eaab99fd13f4d09ad2b60a286fe4e57: Sending DeleteTablet for 3 replicas of tablet 58ccfefc33be4d0ab7d00843293ccb29
I20260809 07:52:32.878655 7396 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ccfefc33be4d0ab7d00843293ccb29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:32 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:48236
I20260809 07:52:32.878719 7110 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ccfefc33be4d0ab7d00843293ccb29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:32 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:40016
I20260809 07:52:32.878793 7252 tablet_service.cc:1514] Processing DeleteTablet for tablet 58ccfefc33be4d0ab7d00843293ccb29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:32 UTC) from {username='kudu', principal='kudu/127.18.71.126@KRBTEST.COM'} at 127.0.0.1:56892
I20260809 07:52:32.879174 7511 tablet_replica.cc:330] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: stopping tablet replica
I20260809 07:52:32.879289 7510 tablet_replica.cc:330] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63: stopping tablet replica
I20260809 07:52:32.879562 7511 raft_consensus.cc:2229] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:32.879572 7510 raft_consensus.cc:2229] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:32.879877 7511 raft_consensus.cc:2258] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:32.879877 7510 raft_consensus.cc:2258] T 58ccfefc33be4d0ab7d00843293ccb29 P 810808e3ab3a4da0913034d195233d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:32.879913 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 7043
I20260809 07:52:32.883886 7511 ts_tablet_manager.cc:1905] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:32.887442 7511 ts_tablet_manager.cc:1918] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:32.887589 7511 log.cc:1196] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/58ccfefc33be4d0ab7d00843293ccb29
I20260809 07:52:32.888015 7511 ts_tablet_manager.cc:1939] T 58ccfefc33be4d0ab7d00843293ccb29 P add76052835c480b9f07bc0fe7cfc209: Deleting consensus metadata
I20260809 07:52:32.888209 7512 tablet_replica.cc:330] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: stopping tablet replica
I20260809 07:52:32.888577 7512 raft_consensus.cc:2229] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 07:52:32.889021 7512 raft_consensus.cc:2258] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:32.889439 6962 catalog_manager.cc:4841] TS add76052835c480b9f07bc0fe7cfc209 (127.18.71.67:35361): tablet 58ccfefc33be4d0ab7d00843293ccb29 (table test-table [id=368ef102cc574470af6614e3817f3263]) successfully deleted
I20260809 07:52:32.890666 7512 ts_tablet_manager.cc:1905] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:32.893088 7512 ts_tablet_manager.cc:1918] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:32.893211 7512 log.cc:1196] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/58ccfefc33be4d0ab7d00843293ccb29
I20260809 07:52:32.893524 7512 ts_tablet_manager.cc:1939] T 58ccfefc33be4d0ab7d00843293ccb29 P d5378d228cb14719ba0b27115aabe9c4: Deleting consensus metadata
I20260809 07:52:32.894605 6962 catalog_manager.cc:4841] TS d5378d228cb14719ba0b27115aabe9c4 (127.18.71.66:33631): tablet 58ccfefc33be4d0ab7d00843293ccb29 (table test-table [id=368ef102cc574470af6614e3817f3263]) successfully deleted
W20260809 07:52:32.894893 6962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:40009 (error 108)
I20260809 07:52:32.895102 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 7184
I20260809 07:52:32.898072 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:32.897656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:40009 (local address 127.0.0.1:40026)
0809 07:52:32.897873 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:32.897956 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
W20260809 07:52:32.898277 6962 catalog_manager.cc:4584] TS 810808e3ab3a4da0913034d195233d63 (127.18.71.65:40009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58ccfefc33be4d0ab7d00843293ccb29: Network error: Client connection negotiation failed: client connection to 127.18.71.65:40009: connect: Connection refused (error 111)
I20260809 07:52:32.903649 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 7328
I20260809 07:52:32.910846 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 6943
2026-08-09T07:52:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2632 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T07:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:32Z Disabled control of system clock
I20260809 07:52:32.945294 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46861
--webserver_interface=127.18.71.126
--webserver_port=0
--builtin_ntp_servers=127.18.71.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:46861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 07:52:33.060032 7517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:33.060285 7517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:33.060320 7517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:33.064123 7517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:33.064179 7517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:33.064194 7517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:33.064210 7517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:33.064224 7517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:33.068941 7517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:46861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:33.069944 7517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:33.071365 7517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:33.080123 7526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.080138 7524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.080138 7523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:33.080258 7517 server_base.cc:1034] running on GCE node
I20260809 07:52:33.081001 7517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:33.082129 7517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:33.083324 7517 hybrid_clock.cc:648] HybridClock initialized: now 1786261953083301 us; error 32 us; skew 500 ppm
I20260809 07:52:33.085210 7517 webserver.cc:460] Webserver started at http://127.18.71.126:43545/ using document root <none> and password file <none>
I20260809 07:52:33.085788 7517 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:33.085883 7517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:33.086129 7517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:33.088182 7517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "0fd8327ce14a412483ca057ad2c11bf1"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "339b3b6e36174c551cde3e4cd208552b"
server_key_iv: "331844d0696e632f02741dca57678dc9"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.088792 7517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "0fd8327ce14a412483ca057ad2c11bf1"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "339b3b6e36174c551cde3e4cd208552b"
server_key_iv: "331844d0696e632f02741dca57678dc9"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.092554 7517 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 07:52:33.094980 7532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.096222 7517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:33.096360 7517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "0fd8327ce14a412483ca057ad2c11bf1"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "339b3b6e36174c551cde3e4cd208552b"
server_key_iv: "331844d0696e632f02741dca57678dc9"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.096503 7517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:33.114990 7517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:33.115708 7517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:33.115914 7517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:33.123075 7517 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:46861
I20260809 07:52:33.123071 7591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:46861 every 8 connection(s)
I20260809 07:52:33.124089 7517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:33.127892 7592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:33.131709 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 7517
I20260809 07:52:33.131845 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:33.132176 18717 external_mini_cluster.cc:1428] Setting key 19b111441c3d667f36f41466f8227f01
I20260809 07:52:33.134321 7592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1: Bootstrap starting.
I20260809 07:52:33.136619 7592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:33.137360 7592 log.cc:824] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:33.139228 7592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1: No bootstrap required, opened a new log
I20260809 07:52:33.142478 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.133823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48234 (local address 127.18.71.126:46861)
0809 07:52:33.134600 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.134618 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.134656 (+ 38us) server_negotiation.cc:407] Connection header received
0809 07:52:33.135299 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.135315 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.135779 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.136123 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:33.136636 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.137786 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.138687 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.139166 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.139342 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.140276 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:33.140281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:33.140294 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:33.140773 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:33.140832 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:33.140854 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:33.140968 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.141133 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.141395 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":143,"threads_started":1}
I20260809 07:52:33.142650 7592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fd8327ce14a412483ca057ad2c11bf1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46861 } }
I20260809 07:52:33.142963 7592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:33.143028 7592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fd8327ce14a412483ca057ad2c11bf1, State: Initialized, Role: FOLLOWER
I20260809 07:52:33.143497 7592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fd8327ce14a412483ca057ad2c11bf1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46861 } }
I20260809 07:52:33.143680 7592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:33.143784 7592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:33.143909 7592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:33.145035 7592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fd8327ce14a412483ca057ad2c11bf1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46861 } }
I20260809 07:52:33.145401 7592 leader_election.cc:302] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fd8327ce14a412483ca057ad2c11bf1; no voters:
I20260809 07:52:33.145740 7592 leader_election.cc:288] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:33.145867 7597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:33.146106 7597 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [term 1 LEADER]: Becoming Leader. State: Replica: 0fd8327ce14a412483ca057ad2c11bf1, State: Running, Role: LEADER
I20260809 07:52:33.146447 7597 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fd8327ce14a412483ca057ad2c11bf1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46861 } }
I20260809 07:52:33.146935 7592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:33.148108 7599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fd8327ce14a412483ca057ad2c11bf1. Latest consensus state: current_term: 1 leader_uuid: "0fd8327ce14a412483ca057ad2c11bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fd8327ce14a412483ca057ad2c11bf1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46861 } } }
I20260809 07:52:33.148237 7599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:33.148598 7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fd8327ce14a412483ca057ad2c11bf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fd8327ce14a412483ca057ad2c11bf1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46861 } } }
I20260809 07:52:33.148754 7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:33.148662 7606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:33.152004 7606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:33.158265 7606 catalog_manager.cc:1347] Generated new cluster ID: 8c097462608f4848806057c307ecdc87
I20260809 07:52:33.158389 7606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:33.169517 7606 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:33.170436 7606 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:33.179744 7606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1: Generated new TSK 0
I20260809 07:52:33.180491 7606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:33.190222 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.65:0
--local_ip_for_outbound_sockets=127.18.71.65
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46861
--builtin_ntp_servers=127.18.71.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 07:52:33.308619 7616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:33.308863 7616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:33.308900 7616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:33.312844 7616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:33.312894 7616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:33.312945 7616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.65
I20260809 07:52:33.317456 7616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.18.71.65
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.65
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:33.318434 7616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:33.319751 7616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:33.329627 7624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.329612 7621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.329595 7622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:33.329689 7616 server_base.cc:1034] running on GCE node
I20260809 07:52:33.330236 7616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:33.330919 7616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:33.332116 7616 hybrid_clock.cc:648] HybridClock initialized: now 1786261953332102 us; error 34 us; skew 500 ppm
I20260809 07:52:33.338090 7616 webserver.cc:460] Webserver started at http://127.18.71.65:45407/ using document root <none> and password file <none>
I20260809 07:52:33.338734 7616 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:33.338832 7616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:33.339085 7616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:33.341313 7616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data/instance:
uuid: "2121adc55f364d0b9cf871dcc4157fd4"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "2ef7bb339cf365090b82b9557e260c52"
server_key_iv: "0f28f5c2ef5eeac87ef63b850884aa93"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.341898 7616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance:
uuid: "2121adc55f364d0b9cf871dcc4157fd4"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "2ef7bb339cf365090b82b9557e260c52"
server_key_iv: "0f28f5c2ef5eeac87ef63b850884aa93"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.345675 7616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:33.348439 7631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.349749 7616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:33.349882 7616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/wal
uuid: "2121adc55f364d0b9cf871dcc4157fd4"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "2ef7bb339cf365090b82b9557e260c52"
server_key_iv: "0f28f5c2ef5eeac87ef63b850884aa93"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.350003 7616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:33.376259 7616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:33.377027 7616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:33.377234 7616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:33.377853 7616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:33.378818 7616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:33.378891 7616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.378965 7616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:33.379010 7616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.388327 7616 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.65:43755
I20260809 07:52:33.388370 7754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.65:43755 every 8 connection(s)
I20260809 07:52:33.389274 7616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/data/info.pb
I20260809 07:52:33.391301 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 7616
I20260809 07:52:33.391455 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-0/wal/instance
I20260809 07:52:33.391757 18717 external_mini_cluster.cc:1428] Setting key 04dd9119b6d94f2321a8937f540c2678
I20260809 07:52:33.394080 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.66:0
--local_ip_for_outbound_sockets=127.18.71.66
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46861
--builtin_ntp_servers=127.18.71.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 07:52:33.400238 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.391126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:58449 (local address 127.18.71.126:46861)
0809 07:52:33.391292 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.391296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.392618 (+ 1322us) server_negotiation.cc:407] Connection header received
0809 07:52:33.394431 (+ 1813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.394434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.394476 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.394551 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:33.395735 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.396260 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.398031 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.398261 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.398336 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.399757 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:33.399760 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:33.399761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:33.399773 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:33.399791 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:33.399799 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:33.399857 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.400066 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.400128 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:52:33.401520 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.391630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46861 (local address 127.18.71.65:58449)
0809 07:52:33.392435 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.392539 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.394192 (+ 1653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.394749 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.394762 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.394876 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:33.395585 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.395608 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.396391 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.396398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.397909 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.397925 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.398411 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.398418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.398647 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.399445 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:33.399449 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:33.399510 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:33.399602 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:33.399891 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:33.399904 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:33.399912 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.400065 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.400388 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":142,"threads_started":1}
I20260809 07:52:33.402746 7755 heartbeater.cc:347] Connected to a master server at 127.18.71.126:46861
I20260809 07:52:33.402992 7755 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:33.403509 7755 heartbeater.cc:511] Master 127.18.71.126:46861 requested a full tablet report, sending...
I20260809 07:52:33.405055 7554 ts_manager.cc:194] Registered new tserver with Master: 2121adc55f364d0b9cf871dcc4157fd4 (127.18.71.65:43755)
I20260809 07:52:33.406188 7554 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.65:58449
W20260809 07:52:33.509043 7758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:33.509292 7758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:33.509330 7758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:33.513296 7758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:33.513357 7758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:33.513455 7758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.66
I20260809 07:52:33.518226 7758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.18.71.66
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.66
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:33.519377 7758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:33.520850 7758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:33.530398 7766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.530503 7763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.530372 7764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:33.530685 7758 server_base.cc:1034] running on GCE node
I20260809 07:52:33.531111 7758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:33.531730 7758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:33.532889 7758 hybrid_clock.cc:648] HybridClock initialized: now 1786261953532875 us; error 24 us; skew 500 ppm
I20260809 07:52:33.534677 7758 webserver.cc:460] Webserver started at http://127.18.71.66:40827/ using document root <none> and password file <none>
I20260809 07:52:33.535243 7758 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:33.535305 7758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:33.535548 7758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:33.537649 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data/instance:
uuid: "b2b55bd74bc340219f66855e31b4a2a6"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "ddc816b7d72898045937cfd8f2fda3ad"
server_key_iv: "d6e2c48e22f18636f9ba0da6b6242004"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.538211 7758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance:
uuid: "b2b55bd74bc340219f66855e31b4a2a6"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "ddc816b7d72898045937cfd8f2fda3ad"
server_key_iv: "d6e2c48e22f18636f9ba0da6b6242004"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.541819 7758 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:33.544094 7772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.545370 7758 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:33.545487 7758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal
uuid: "b2b55bd74bc340219f66855e31b4a2a6"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "ddc816b7d72898045937cfd8f2fda3ad"
server_key_iv: "d6e2c48e22f18636f9ba0da6b6242004"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.545603 7758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:33.569885 7758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:33.570672 7758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:33.570883 7758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:33.571513 7758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:33.572569 7758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:33.572619 7758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.572683 7758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:33.572719 7758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.581903 7758 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.66:37443
I20260809 07:52:33.581936 7897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.66:37443 every 8 connection(s)
I20260809 07:52:33.582830 7758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/data/info.pb
I20260809 07:52:33.591822 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.584705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.66:42783 (local address 127.18.71.126:46861)
0809 07:52:33.584843 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.584848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.585945 (+ 1097us) server_negotiation.cc:407] Connection header received
0809 07:52:33.587283 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.587286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.587331 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.587405 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:33.588285 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.588890 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.589985 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.590211 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.590288 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.591418 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:33.591421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:33.591423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:33.591434 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:33.591451 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:33.591458 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:33.591513 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.591660 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.591708 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 07:52:33.592480 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 7758
I20260809 07:52:33.592628 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal/instance
I20260809 07:52:33.592970 18717 external_mini_cluster.cc:1428] Setting key f7e23c9dfd02b22e731de5f2d8d78987
I20260809 07:52:33.592989 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.585140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46861 (local address 127.18.71.66:42783)
0809 07:52:33.585787 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.585827 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.587097 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.587576 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.587583 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.587681 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:33.588161 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.588173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.588996 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.588998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.589874 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.589883 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.590344 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.590346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.590493 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.591172 (+ 679us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:33.591174 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:33.591214 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:33.591279 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:33.591532 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:33.591539 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:33.591543 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.591648 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.591889 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":112,"threads_started":1}
I20260809 07:52:33.594226 7898 heartbeater.cc:347] Connected to a master server at 127.18.71.126:46861
I20260809 07:52:33.594482 7898 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:33.595096 7898 heartbeater.cc:511] Master 127.18.71.126:46861 requested a full tablet report, sending...
I20260809 07:52:33.595330 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.18.71.67:0
--local_ip_for_outbound_sockets=127.18.71.67
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46861
--builtin_ntp_servers=127.18.71.84:40929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 07:52:33.596227 7553 ts_manager.cc:194] Registered new tserver with Master: b2b55bd74bc340219f66855e31b4a2a6 (127.18.71.66:37443)
I20260809 07:52:33.596874 7553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.66:42783
W20260809 07:52:33.712831 7901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:33.713143 7901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:33.713230 7901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:33.717167 7901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:33.717242 7901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:33.717329 7901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.18.71.67
I20260809 07:52:33.722047 7901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:40929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.18.71.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.18.71.67
--webserver_port=0
--tserver_master_addrs=127.18.71.126:46861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.18.71.67
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:33.723060 7901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:33.724439 7901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:52:33.734308 7901 server_base.cc:1034] running on GCE node
W20260809 07:52:33.734297 7909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.734326 7907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:33.734326 7906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:33.734864 7901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:33.735502 7901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:33.736693 7901 hybrid_clock.cc:648] HybridClock initialized: now 1786261953736668 us; error 55 us; skew 500 ppm
I20260809 07:52:33.738441 7901 webserver.cc:460] Webserver started at http://127.18.71.67:35609/ using document root <none> and password file <none>
I20260809 07:52:33.739043 7901 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:33.739138 7901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:33.739382 7901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:33.741513 7901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data/instance:
uuid: "c375b7593e72437e94217b2672389d51"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "e4c1f0b8c9f233e3370fa7b4bafe09e1"
server_key_iv: "afe80a66fa92c7aca74e07a83c49220f"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.742133 7901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance:
uuid: "c375b7593e72437e94217b2672389d51"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "e4c1f0b8c9f233e3370fa7b4bafe09e1"
server_key_iv: "afe80a66fa92c7aca74e07a83c49220f"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.745817 7901 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:33.748376 7915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.749642 7901 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:33.749781 7901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal
uuid: "c375b7593e72437e94217b2672389d51"
format_stamp: "Formatted at 2026-08-09 07:52:33 on dist-test-slave-10pc"
server_key: "e4c1f0b8c9f233e3370fa7b4bafe09e1"
server_key_iv: "afe80a66fa92c7aca74e07a83c49220f"
server_key_version: "encryptionkey@0"
I20260809 07:52:33.749894 7901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:33.763620 7901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:33.764284 7901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:33.764551 7901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:33.765151 7901 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 07:52:33.766083 7901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 07:52:33.766160 7901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.766235 7901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 07:52:33.766280 7901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:33.775434 7901 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.67:46011
I20260809 07:52:33.775476 8039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.67:46011 every 8 connection(s)
I20260809 07:52:33.776310 7901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/data/info.pb
I20260809 07:52:33.782853 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 7901
I20260809 07:52:33.782982 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal/instance
I20260809 07:52:33.783294 18717 external_mini_cluster.cc:1428] Setting key ceebda92e3d819c91d258d9e90d423cb
I20260809 07:52:33.785496 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.778126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.67:57245 (local address 127.18.71.126:46861)
0809 07:52:33.778261 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.778265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.779303 (+ 1038us) server_negotiation.cc:407] Connection header received
0809 07:52:33.780658 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.780660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.780705 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.780778 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:33.781602 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.782129 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.783339 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.783615 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.783734 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.784997 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:33.785000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:33.785002 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:33.785012 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:33.785032 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:33.785040 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:33.785098 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.785268 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.785342 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:52:33.786312 8042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.778521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46861 (local address 127.18.71.67:57245)
0809 07:52:33.779148 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.779187 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.780439 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.780905 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.780911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.780990 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:52:33.781471 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.781485 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.782235 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.782237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.783215 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.783224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.783774 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.783779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.783979 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.784710 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 07:52:33.784714 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:52:33.784772 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 07:52:33.784850 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:52:33.785124 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:52:33.785133 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:52:33.785139 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.785265 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.785551 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":118,"threads_started":1}
I20260809 07:52:33.787416 8040 heartbeater.cc:347] Connected to a master server at 127.18.71.126:46861
I20260809 07:52:33.787662 8040 heartbeater.cc:464] Registering TS with master...
I20260809 07:52:33.788144 8040 heartbeater.cc:511] Master 127.18.71.126:46861 requested a full tablet report, sending...
I20260809 07:52:33.789141 7553 ts_manager.cc:194] Registered new tserver with Master: c375b7593e72437e94217b2672389d51 (127.18.71.67:46011)
I20260809 07:52:33.789692 7553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.18.71.67:57245
I20260809 07:52:33.797186 18717 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 07:52:33.802636 7595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.799043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48250 (local address 127.18.71.126:46861)
0809 07:52:33.799195 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.799230 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.799387 (+ 157us) server_negotiation.cc:407] Connection header received
0809 07:52:33.799463 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.799465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.799501 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.799564 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:33.799789 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.800385 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.801229 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.801431 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.801505 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.802322 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:33.802325 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:33.802327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:33.802337 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:33.802353 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:33.802360 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:33.802417 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.802494 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.802537 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 07:52:33.805274 7553 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48250:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 07:52:33.808200 7553 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 07:52:33.821935 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.817123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:43755 (local address 127.0.0.1:37454)
0809 07:52:33.817392 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.817406 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.817489 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.818237 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.818248 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.818276 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:33.818418 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.818424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.819446 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.819451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.820715 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.820724 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.821568 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.821570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.821646 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.821675 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.821751 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.821829 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":90,"threads_started":1}
I20260809 07:52:33.822034 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.817312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37454 (local address 127.18.71.65:43755)
0809 07:52:33.817791 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.817804 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.817823 (+ 19us) server_negotiation.cc:407] Connection header received
0809 07:52:33.817897 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.817901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.818065 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.818227 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:33.818544 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.819328 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.820835 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.821373 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.821546 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.821741 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.821791 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.821878 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":144,"threads_started":1}
I20260809 07:52:33.822799 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.816429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:46011 (local address 127.0.0.1:58100)
0809 07:52:33.817083 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.817098 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.817176 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.818342 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.818345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.818358 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:33.818421 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.818425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.819979 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.819983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.821332 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.821344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.822385 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.822390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.822497 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.822521 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.822611 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.822676 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":50,"thread_start_us":143,"threads_started":1}
I20260809 07:52:33.822916 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.817555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.18.71.67:46011)
0809 07:52:33.817927 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.817937 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.817957 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:52:33.818016 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.818019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.818190 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.818579 (+ 389us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:33.818741 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.819839 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.821483 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.822263 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.822370 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.822530 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.822662 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.822750 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":179,"threads_started":1}
I20260809 07:52:33.824266 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.816363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:37443 (local address 127.0.0.1:60884)
0809 07:52:33.816832 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.816876 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.816993 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.818397 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.818399 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.818416 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:33.818527 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.818533 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.819979 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.819983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.821332 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.821344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.823901 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.823905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.823991 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.824012 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.824075 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.824147 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":143,"thread_start_us":93,"threads_started":1}
I20260809 07:52:33.824379 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.816514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60884 (local address 127.18.71.66:37443)
0809 07:52:33.817737 (+ 1223us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.817749 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.817773 (+ 24us) server_negotiation.cc:407] Connection header received
0809 07:52:33.817856 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.817861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.818066 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.818227 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:33.818653 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.819840 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.822258 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.823777 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.823897 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.824072 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.824132 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.824219 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108,"thread_start_us":103,"threads_started":1}
I20260809 07:52:33.825572 7968 tablet_service.cc:1467] Processing CreateTablet for tablet 22079db56dac46d4a9624d96a18e6a25 (DEFAULT_TABLE table=test-table [id=926814e361f54c499393df666f079b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:33.825448 7686 tablet_service.cc:1467] Processing CreateTablet for tablet 22079db56dac46d4a9624d96a18e6a25 (DEFAULT_TABLE table=test-table [id=926814e361f54c499393df666f079b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:33.826614 7686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22079db56dac46d4a9624d96a18e6a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:33.826619 7968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22079db56dac46d4a9624d96a18e6a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:33.826741 7828 tablet_service.cc:1467] Processing CreateTablet for tablet 22079db56dac46d4a9624d96a18e6a25 (DEFAULT_TABLE table=test-table [id=926814e361f54c499393df666f079b84]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 07:52:33.827751 7828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22079db56dac46d4a9624d96a18e6a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:33.833778 8058 tablet_bootstrap.cc:492] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Bootstrap starting.
I20260809 07:52:33.834815 8059 tablet_bootstrap.cc:492] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Bootstrap starting.
I20260809 07:52:33.836090 8058 tablet_bootstrap.cc:654] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:33.836678 8059 tablet_bootstrap.cc:654] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:33.836982 8058 log.cc:824] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:33.837549 8059 log.cc:824] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:33.837759 8060 tablet_bootstrap.cc:492] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4: Bootstrap starting.
I20260809 07:52:33.839215 8059 tablet_bootstrap.cc:492] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: No bootstrap required, opened a new log
I20260809 07:52:33.839399 8060 tablet_bootstrap.cc:654] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:33.839416 8059 ts_tablet_manager.cc:1397] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:52:33.840022 8060 log.cc:824] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:33.841464 8058 tablet_bootstrap.cc:492] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: No bootstrap required, opened a new log
I20260809 07:52:33.841660 8058 ts_tablet_manager.cc:1397] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260809 07:52:33.841933 8060 tablet_bootstrap.cc:492] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4: No bootstrap required, opened a new log
I20260809 07:52:33.842113 8060 ts_tablet_manager.cc:1397] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 07:52:33.843502 8059 raft_consensus.cc:348] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.843762 8059 raft_consensus.cc:374] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:33.843822 8059 raft_consensus.cc:729] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c375b7593e72437e94217b2672389d51, State: Initialized, Role: FOLLOWER
I20260809 07:52:33.844326 8059 consensus_queue.cc:260] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.845165 8058 raft_consensus.cc:348] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.845362 8059 ts_tablet_manager.cc:1428] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260809 07:52:33.845381 8058 raft_consensus.cc:374] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:33.845436 8058 raft_consensus.cc:729] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b55bd74bc340219f66855e31b4a2a6, State: Initialized, Role: FOLLOWER
I20260809 07:52:33.845721 8040 heartbeater.cc:503] Master 127.18.71.126:46861 was elected leader, sending a full tablet report...
I20260809 07:52:33.845870 8058 consensus_queue.cc:260] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.846571 7898 heartbeater.cc:503] Master 127.18.71.126:46861 was elected leader, sending a full tablet report...
I20260809 07:52:33.846458 8060 raft_consensus.cc:348] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.846738 8060 raft_consensus.cc:374] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:33.846786 8060 raft_consensus.cc:729] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2121adc55f364d0b9cf871dcc4157fd4, State: Initialized, Role: FOLLOWER
I20260809 07:52:33.847054 8058 ts_tablet_manager.cc:1428] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 07:52:33.847193 8060 consensus_queue.cc:260] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.848140 7755 heartbeater.cc:503] Master 127.18.71.126:46861 was elected leader, sending a full tablet report...
I20260809 07:52:33.848152 8060 ts_tablet_manager.cc:1428] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260809 07:52:33.854354 8067 raft_consensus.cc:482] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 07:52:33.854504 8067 raft_consensus.cc:504] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.855479 8067 leader_election.cc:288] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b2b55bd74bc340219f66855e31b4a2a6 (127.18.71.66:37443), c375b7593e72437e94217b2672389d51 (127.18.71.67:46011)
I20260809 07:52:33.859211 7757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.855693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.66:37443 (local address 127.18.71.65:45973)
0809 07:52:33.855790 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.855807 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.855891 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.856173 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.856177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.856213 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:33.856336 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.856346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.857184 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.857187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.858076 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.858086 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.858792 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.858797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.858906 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.858935 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.859006 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.859086 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 07:52:33.859261 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.855770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:45973 (local address 127.18.71.66:37443)
0809 07:52:33.855874 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.855879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.855895 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:52:33.856003 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.856006 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.856052 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.856153 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:33.856478 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.857085 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.858208 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.858677 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.858766 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.858802 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.859044 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.859137 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 07:52:33.859402 8057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.855770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.18.71.65:47741 (local address 127.18.71.67:46011)
0809 07:52:33.855874 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.855879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.856150 (+ 271us) server_negotiation.cc:407] Connection header received
0809 07:52:33.856254 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.856257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.856304 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.856384 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 07:52:33.856698 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.857343 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.858397 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.858834 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.858947 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.859010 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.859211 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.859291 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 07:52:33.859714 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.855693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.67:46011 (local address 127.18.71.65:47741)
0809 07:52:33.856054 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.856068 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:52:33.856159 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:52:33.856422 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:52:33.856425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:52:33.856443 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 07:52:33.856576 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.856582 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.857453 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.857456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.858276 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:52:33.858286 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.858959 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.858962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:52:33.859080 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.859101 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 07:52:33.859179 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 07:52:33.859242 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":101,"thread_start_us":74,"threads_started":1}
I20260809 07:52:33.860103 7849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22079db56dac46d4a9624d96a18e6a25" candidate_uuid: "2121adc55f364d0b9cf871dcc4157fd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b55bd74bc340219f66855e31b4a2a6" is_pre_election: true
I20260809 07:52:33.860370 7849 raft_consensus.cc:2454] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2121adc55f364d0b9cf871dcc4157fd4 in term 0.
I20260809 07:52:33.860888 7632 leader_election.cc:302] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2121adc55f364d0b9cf871dcc4157fd4, b2b55bd74bc340219f66855e31b4a2a6; no voters:
I20260809 07:52:33.861166 8067 raft_consensus.cc:2781] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 07:52:33.861163 7989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22079db56dac46d4a9624d96a18e6a25" candidate_uuid: "2121adc55f364d0b9cf871dcc4157fd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c375b7593e72437e94217b2672389d51" is_pre_election: true
I20260809 07:52:33.861264 8067 raft_consensus.cc:482] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 07:52:33.861313 8067 raft_consensus.cc:3037] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:33.861430 7989 raft_consensus.cc:2454] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2121adc55f364d0b9cf871dcc4157fd4 in term 0.
I20260809 07:52:33.862421 8067 raft_consensus.cc:504] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.862825 8067 leader_election.cc:288] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [CANDIDATE]: Term 1 election: Requested vote from peers b2b55bd74bc340219f66855e31b4a2a6 (127.18.71.66:37443), c375b7593e72437e94217b2672389d51 (127.18.71.67:46011)
I20260809 07:52:33.863101 7849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22079db56dac46d4a9624d96a18e6a25" candidate_uuid: "2121adc55f364d0b9cf871dcc4157fd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2b55bd74bc340219f66855e31b4a2a6"
I20260809 07:52:33.863229 7849 raft_consensus.cc:3037] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:33.864183 7849 raft_consensus.cc:2454] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2121adc55f364d0b9cf871dcc4157fd4 in term 1.
I20260809 07:52:33.864493 7632 leader_election.cc:302] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2121adc55f364d0b9cf871dcc4157fd4, b2b55bd74bc340219f66855e31b4a2a6; no voters:
I20260809 07:52:33.864454 7989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22079db56dac46d4a9624d96a18e6a25" candidate_uuid: "2121adc55f364d0b9cf871dcc4157fd4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c375b7593e72437e94217b2672389d51"
I20260809 07:52:33.864604 7989 raft_consensus.cc:3037] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:33.864670 8067 raft_consensus.cc:2781] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:33.864908 8067 raft_consensus.cc:686] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [term 1 LEADER]: Becoming Leader. State: Replica: 2121adc55f364d0b9cf871dcc4157fd4, State: Running, Role: LEADER
I20260809 07:52:33.865545 8067 consensus_queue.cc:237] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } }
I20260809 07:52:33.865872 7989 raft_consensus.cc:2454] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2121adc55f364d0b9cf871dcc4157fd4 in term 1.
I20260809 07:52:33.868901 7554 catalog_manager.cc:5495] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2121adc55f364d0b9cf871dcc4157fd4 (127.18.71.65). New cstate: current_term: 1 leader_uuid: "2121adc55f364d0b9cf871dcc4157fd4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2121adc55f364d0b9cf871dcc4157fd4" member_type: VOTER last_known_addr { host: "127.18.71.65" port: 43755 } health_report { overall_health: HEALTHY } } }
I20260809 07:52:33.883147 8056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.879832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.18.71.65:43755)
0809 07:52:33.879968 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:33.879972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:33.879999 (+ 27us) server_negotiation.cc:407] Connection header received
0809 07:52:33.880149 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:33.880151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:33.880202 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:33.880284 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:52:33.880528 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.881038 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.881868 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:33.882071 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:33.882143 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:33.882384 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:52:33.882779 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:52:33.882868 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:33.882920 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 07:52:33.882962 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 07:52:33.888772 7849 raft_consensus.cc:1261] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 1 FOLLOWER]: Refusing update from remote peer 2121adc55f364d0b9cf871dcc4157fd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:33.888780 7989 raft_consensus.cc:1261] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 1 FOLLOWER]: Refusing update from remote peer 2121adc55f364d0b9cf871dcc4157fd4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:52:33.889472 8067 consensus_queue.cc:1035] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c375b7593e72437e94217b2672389d51" member_type: VOTER last_known_addr { host: "127.18.71.67" port: 46011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:52:33.889671 8069 consensus_queue.cc:1035] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2b55bd74bc340219f66855e31b4a2a6" member_type: VOTER last_known_addr { host: "127.18.71.66" port: 37443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 07:52:33.890415 7756 tablet.cc:2369] T 22079db56dac46d4a9624d96a18e6a25 P 2121adc55f364d0b9cf871dcc4157fd4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:52:33.898022 8073 mvcc.cc:205] Tried to move back new op lower bound from 7316528963121479680 to 7316528963036790784. Current Snapshot: MvccSnapshot[applied={T|T < 7316528963121479680}]
I20260809 07:52:33.906809 7554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:33.906955 7554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48250:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 07:52:33.909335 7554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fd8327ce14a412483ca057ad2c11bf1: Sending DeleteTablet for 3 replicas of tablet 22079db56dac46d4a9624d96a18e6a25
I20260809 07:52:33.910099 7828 tablet_service.cc:1514] Processing DeleteTablet for tablet 22079db56dac46d4a9624d96a18e6a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:33 UTC) from {username='slave'} at 127.0.0.1:60884
I20260809 07:52:33.910200 7968 tablet_service.cc:1514] Processing DeleteTablet for tablet 22079db56dac46d4a9624d96a18e6a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:33 UTC) from {username='slave'} at 127.0.0.1:58100
I20260809 07:52:33.910349 7686 tablet_service.cc:1514] Processing DeleteTablet for tablet 22079db56dac46d4a9624d96a18e6a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 07:52:33 UTC) from {username='slave'} at 127.0.0.1:37454
I20260809 07:52:33.910635 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 7616
I20260809 07:52:33.913868 8083 tablet_replica.cc:330] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: stopping tablet replica
I20260809 07:52:33.914256 8083 raft_consensus.cc:2229] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:33.914583 8083 raft_consensus.cc:2258] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:33.914615 8084 tablet_replica.cc:330] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: stopping tablet replica
I20260809 07:52:33.914873 8084 raft_consensus.cc:2229] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 07:52:33.915169 8084 raft_consensus.cc:2258] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 07:52:33.915972 8083 ts_tablet_manager.cc:1905] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:33.916711 8084 ts_tablet_manager.cc:1905] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 07:52:33.918305 8083 ts_tablet_manager.cc:1918] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:33.918421 8083 log.cc:1196] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-1/wal/wals/22079db56dac46d4a9624d96a18e6a25
I20260809 07:52:33.918752 8083 ts_tablet_manager.cc:1939] T 22079db56dac46d4a9624d96a18e6a25 P b2b55bd74bc340219f66855e31b4a2a6: Deleting consensus metadata
I20260809 07:52:33.919904 7533 catalog_manager.cc:4841] TS b2b55bd74bc340219f66855e31b4a2a6 (127.18.71.66:37443): tablet 22079db56dac46d4a9624d96a18e6a25 (table test-table [id=926814e361f54c499393df666f079b84]) successfully deleted
I20260809 07:52:33.920053 8084 ts_tablet_manager.cc:1918] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 07:52:33.920156 8084 log.cc:1196] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Deleting WAL directory at /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786261695308648-18717-0/minicluster-data/ts-2/wal/wals/22079db56dac46d4a9624d96a18e6a25
I20260809 07:52:33.920496 8084 ts_tablet_manager.cc:1939] T 22079db56dac46d4a9624d96a18e6a25 P c375b7593e72437e94217b2672389d51: Deleting consensus metadata
I20260809 07:52:33.921759 7537 catalog_manager.cc:4841] TS c375b7593e72437e94217b2672389d51 (127.18.71.67:46011): tablet 22079db56dac46d4a9624d96a18e6a25 (table test-table [id=926814e361f54c499393df666f079b84]) successfully deleted
W20260809 07:52:33.922654 7537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.18.71.65:43755 (error 108)
I20260809 07:52:33.922709 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 7758
I20260809 07:52:33.924575 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:33.924194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.65:43755 (local address 127.0.0.1:37470)
0809 07:52:33.924365 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 07:52:33.924470 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.65:43755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 07:52:33.925379 7537 catalog_manager.cc:4584] TS 2121adc55f364d0b9cf871dcc4157fd4 (127.18.71.65:43755): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 22079db56dac46d4a9624d96a18e6a25: Network error: Client connection negotiation failed: client connection to 127.18.71.65:43755: connect: Connection refused (error 111)
I20260809 07:52:33.930387 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 7901
I20260809 07:52:33.939474 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 7517
2026-08-09T07:52:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1026 ms)
[----------] 6 tests from EncryptionPolicyTest (10902 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:34 dist-test-slave-10pc krb5kdc[8089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:34 dist-test-slave-10pc krb5kdc[8089](info): set up 2 sockets
Aug 09 07:52:34 dist-test-slave-10pc krb5kdc[8089](info): commencing operation
krb5kdc: starting...
W20260809 07:52:35.024578 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:35 dist-test-slave-10pc krb5kdc[8089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:35.151960 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34437
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:35.266685 8102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:35.266994 8102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:35.267072 8102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:35.271126 8102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:35.271217 8102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:35.271257 8102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:35.271315 8102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:35.271353 8102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:35.276554 8102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34437
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:35.277701 8102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:35.278743 8102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:35.287705 8107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:35.287752 8110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:35.287757 8108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:35.288132 8102 server_base.cc:1034] running on GCE node
I20260809 07:52:35.288697 8102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:35.289722 8102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:35.290890 8102 hybrid_clock.cc:648] HybridClock initialized: now 1786261955290857 us; error 45 us; skew 500 ppm
Aug 09 07:52:35 dist-test-slave-10pc krb5kdc[8089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261955, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:35.293751 8102 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:35.294658 8102 webserver.cc:460] Webserver started at http://127.0.0.1:33705/ using document root <none> and password file <none>
I20260809 07:52:35.295230 8102 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:35.295292 8102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:35.295508 8102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:35.297597 8102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "90f37ce297fa4a4f84d111bfccff9293"
format_stamp: "Formatted at 2026-08-09 07:52:35 on dist-test-slave-10pc"
server_key: "751209b7141a09be4cc577516b12b217"
server_key_iv: "cbfc1b912ee575cb0ce625aab430c164"
server_key_version: "encryptionkey@0"
I20260809 07:52:35.298199 8102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "90f37ce297fa4a4f84d111bfccff9293"
format_stamp: "Formatted at 2026-08-09 07:52:35 on dist-test-slave-10pc"
server_key: "751209b7141a09be4cc577516b12b217"
server_key_iv: "cbfc1b912ee575cb0ce625aab430c164"
server_key_version: "encryptionkey@0"
I20260809 07:52:35.301862 8102 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:35.304234 8119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:35.305527 8102 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 07:52:35.305672 8102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "90f37ce297fa4a4f84d111bfccff9293"
format_stamp: "Formatted at 2026-08-09 07:52:35 on dist-test-slave-10pc"
server_key: "751209b7141a09be4cc577516b12b217"
server_key_iv: "cbfc1b912ee575cb0ce625aab430c164"
server_key_version: "encryptionkey@0"
I20260809 07:52:35.305784 8102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:35.330058 8102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:35.333145 8102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:35.333374 8102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:35.340490 8102 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34437
I20260809 07:52:35.340579 8176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34437 every 8 connection(s)
I20260809 07:52:35.341543 8102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:35.344374 8177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:35.349241 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8102
I20260809 07:52:35.349395 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:35.349710 18717 external_mini_cluster.cc:1428] Setting key 5f38239d3e30239466ef5d7b4138983d
I20260809 07:52:35.350342 8177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293: Bootstrap starting.
I20260809 07:52:35.352811 8177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:35.353832 8177 log.cc:824] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:35.355960 8177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293: No bootstrap required, opened a new log
Aug 09 07:52:35 dist-test-slave-10pc krb5kdc[8089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261955, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 07:52:35.360365 8177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90f37ce297fa4a4f84d111bfccff9293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34437 } }
I20260809 07:52:35.360704 8177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:35.360816 8177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f37ce297fa4a4f84d111bfccff9293, State: Initialized, Role: FOLLOWER
I20260809 07:52:35.361266 8177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90f37ce297fa4a4f84d111bfccff9293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34437 } }
I20260809 07:52:35.361420 8177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:35.361510 8177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:35.361636 8177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:35.362557 8177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90f37ce297fa4a4f84d111bfccff9293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34437 } }
I20260809 07:52:35.362897 8177 leader_election.cc:302] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90f37ce297fa4a4f84d111bfccff9293; no voters:
I20260809 07:52:35.363255 8177 leader_election.cc:288] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:35.363320 8182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:35.363535 8182 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [term 1 LEADER]: Becoming Leader. State: Replica: 90f37ce297fa4a4f84d111bfccff9293, State: Running, Role: LEADER
I20260809 07:52:35.363883 8182 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90f37ce297fa4a4f84d111bfccff9293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34437 } }
I20260809 07:52:35.364619 8177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:35.365339 8184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90f37ce297fa4a4f84d111bfccff9293. Latest consensus state: current_term: 1 leader_uuid: "90f37ce297fa4a4f84d111bfccff9293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90f37ce297fa4a4f84d111bfccff9293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34437 } } }
I20260809 07:52:35.365489 8184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:35.365756 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:35.351138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38407 (local address 127.0.0.1:34437)
0809 07:52:35.351546 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:35.351571 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:35.351608 (+ 37us) server_negotiation.cc:407] Connection header received
0809 07:52:35.352246 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:35.352262 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:35.352688 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:35.353027 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:35.353974 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:35.354999 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:35.356140 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:35.356685 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:35.356817 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:35.359626 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:35.359630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:35.359644 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:35.359670 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:35.361705 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:35.362251 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:35.362257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:35.362263 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:35.362385 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:35.362795 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:35.362803 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:35.362804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:35.363269 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:35.364126 (+ 857us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:35.364356 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 07:52:35.364671 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":131,"threads_started":1}
I20260809 07:52:35.365860 8183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90f37ce297fa4a4f84d111bfccff9293" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90f37ce297fa4a4f84d111bfccff9293" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34437 } } }
I20260809 07:52:35.365967 8183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:35.369122 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 07:52:35.369876 8197 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:35.369973 8197 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:35.370064 8189 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:35.371024 8189 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:35.378211 8189 catalog_manager.cc:1347] Generated new cluster ID: eeef412e857643d7a13493c29d35d42d
I20260809 07:52:35.378299 8189 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:35.380543 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:35.371424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38979 (local address 127.0.0.1:34437)
0809 07:52:35.371574 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:35.371578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:35.371880 (+ 302us) server_negotiation.cc:407] Connection header received
0809 07:52:35.372076 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:35.372080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:35.372122 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:35.372170 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:35.373195 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:35.373744 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:35.374615 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:35.374870 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:35.375009 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:35.376698 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:35.376702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:35.376704 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:35.376729 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:35.378664 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:35.379163 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:35.379166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:35.379168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:35.379250 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:35.379596 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:35.379600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:35.379602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:35.379773 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:35.379930 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:35.380219 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 07:52:35.380312 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 07:52:35.395069 8189 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:35.396291 8189 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:35.401760 8189 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90f37ce297fa4a4f84d111bfccff9293: Generated new TSK 0
I20260809 07:52:35.402379 8189 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:35.410765 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8102
W20260809 07:52:35.419750 8087 connection.cc:535] client connection to 127.0.0.1:34437 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34437): Connection reset by peer (error 104)
2026-08-09T07:52:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1472 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:35 dist-test-slave-10pc krb5kdc[8210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:35 dist-test-slave-10pc krb5kdc[8210](info): set up 2 sockets
Aug 09 07:52:35 dist-test-slave-10pc krb5kdc[8210](info): commencing operation
krb5kdc: starting...
W20260809 07:52:36.492357 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:36 dist-test-slave-10pc krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:36.635682 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42035
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:36.750885 8223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:36.751145 8223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:36.751187 8223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:36.755193 8223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:36.755249 8223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:36.755268 8223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:36.755285 8223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:36.755340 8223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:36.760422 8223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42035
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:36.761508 8223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:36.762630 8223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:36.771297 8229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:36.771387 8232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:36.771296 8228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:36.771620 8223 server_base.cc:1034] running on GCE node
I20260809 07:52:36.772289 8223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:36.773433 8223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:36.774631 8223 hybrid_clock.cc:648] HybridClock initialized: now 1786261956774618 us; error 27 us; skew 500 ppm
Aug 09 07:52:36 dist-test-slave-10pc krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261956, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:36.777467 8223 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:36.778330 8223 webserver.cc:460] Webserver started at http://127.0.0.1:38333/ using document root <none> and password file <none>
I20260809 07:52:36.778888 8223 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:36.778981 8223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:36.779241 8223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:36.781421 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "2288ffd2332640c8b1d55261bd6ba704"
format_stamp: "Formatted at 2026-08-09 07:52:36 on dist-test-slave-10pc"
server_key: "e9662ce2e227170530daab5c20e0695f"
server_key_iv: "458c4117af79aaad1eb1446f62782bf2"
server_key_version: "encryptionkey@0"
I20260809 07:52:36.782015 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "2288ffd2332640c8b1d55261bd6ba704"
format_stamp: "Formatted at 2026-08-09 07:52:36 on dist-test-slave-10pc"
server_key: "e9662ce2e227170530daab5c20e0695f"
server_key_iv: "458c4117af79aaad1eb1446f62782bf2"
server_key_version: "encryptionkey@0"
I20260809 07:52:36.785670 8223 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:36.787954 8239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:36.789217 8223 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:36.789354 8223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "2288ffd2332640c8b1d55261bd6ba704"
format_stamp: "Formatted at 2026-08-09 07:52:36 on dist-test-slave-10pc"
server_key: "e9662ce2e227170530daab5c20e0695f"
server_key_iv: "458c4117af79aaad1eb1446f62782bf2"
server_key_version: "encryptionkey@0"
I20260809 07:52:36.789461 8223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:36.823423 8223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:36.827013 8223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:36.827236 8223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:36.834677 8223 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42035
I20260809 07:52:36.834707 8298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42035 every 8 connection(s)
I20260809 07:52:36.835682 8223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:36.838642 8299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:36.843562 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8223
I20260809 07:52:36.843716 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:36.844033 18717 external_mini_cluster.cc:1428] Setting key c34c06c8c80d3d2f1af081760aca4375
I20260809 07:52:36.844537 8299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704: Bootstrap starting.
I20260809 07:52:36.847028 8299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:36.847747 8299 log.cc:824] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:36.849640 8299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704: No bootstrap required, opened a new log
I20260809 07:52:36.853623 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:36.845550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57711 (local address 127.0.0.1:42035)
0809 07:52:36.846004 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:36.846019 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:36.846060 (+ 41us) server_negotiation.cc:407] Connection header received
0809 07:52:36.846689 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:36.846706 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:36.847028 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:36.847400 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:36.847842 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:36.848913 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:36.849861 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:36.850558 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:36.850735 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:36.851559 (+ 824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:36.851565 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:36.851581 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:36.852066 (+ 485us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:36.852145 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:36.852178 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:36.852332 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:36.852489 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 07:52:36.852648 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":143,"threads_started":1}
I20260809 07:52:36.854127 8299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2288ffd2332640c8b1d55261bd6ba704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42035 } }
I20260809 07:52:36.854389 8299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:36.854491 8299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2288ffd2332640c8b1d55261bd6ba704, State: Initialized, Role: FOLLOWER
I20260809 07:52:36.854969 8299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2288ffd2332640c8b1d55261bd6ba704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42035 } }
I20260809 07:52:36.855095 8299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:36.855178 8299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:36.855299 8299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:36.856343 8299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2288ffd2332640c8b1d55261bd6ba704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42035 } }
I20260809 07:52:36.856660 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:36.856739 8299 leader_election.cc:302] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2288ffd2332640c8b1d55261bd6ba704; no voters:
I20260809 07:52:36.857097 8299 leader_election.cc:288] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:36.857169 8304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:36.857401 8304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [term 1 LEADER]: Becoming Leader. State: Replica: 2288ffd2332640c8b1d55261bd6ba704, State: Running, Role: LEADER
I20260809 07:52:36.857751 8304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2288ffd2332640c8b1d55261bd6ba704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42035 } }
I20260809 07:52:36.858413 8299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:36.859640 8306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2288ffd2332640c8b1d55261bd6ba704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2288ffd2332640c8b1d55261bd6ba704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42035 } } }
I20260809 07:52:36.859642 8305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2288ffd2332640c8b1d55261bd6ba704. Latest consensus state: current_term: 1 leader_uuid: "2288ffd2332640c8b1d55261bd6ba704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2288ffd2332640c8b1d55261bd6ba704" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42035 } } }
I20260809 07:52:36.859788 8306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:36.859804 8305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:36.860225 8318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:36.863536 8318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:36.864707 8302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:36.858911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45307 (local address 127.0.0.1:42035)
0809 07:52:36.859024 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:36.859028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:36.860042 (+ 1014us) server_negotiation.cc:407] Connection header received
0809 07:52:36.860218 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:36.860221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:36.860271 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:36.860357 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:36.860782 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:36.861536 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:36.862699 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:36.863025 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:36.863128 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:36.864151 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:36.864155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:36.864158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:36.864171 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:36.864200 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:36.864214 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:36.864276 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:36.864524 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 07:52:36.864572 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 07:52:36.869766 8318 catalog_manager.cc:1347] Generated new cluster ID: c99b5164864843a8935b9477c49c9f55
I20260809 07:52:36.869833 8318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:36.908578 8318 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:36.909574 8318 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:36.924355 8318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2288ffd2332640c8b1d55261bd6ba704: Generated new TSK 0
I20260809 07:52:36.925155 8318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:36.936715 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8223
W20260809 07:52:36.944722 8208 connection.cc:535] client connection to 127.0.0.1:42035 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42035): Connection reset by peer (error 104)
2026-08-09T07:52:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1524 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:36 dist-test-slave-10pc krb5kdc[8330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:36 dist-test-slave-10pc krb5kdc[8330](info): set up 2 sockets
Aug 09 07:52:37 dist-test-slave-10pc krb5kdc[8330](info): commencing operation
krb5kdc: starting...
W20260809 07:52:38.022085 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:38 dist-test-slave-10pc krb5kdc[8330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:38.150987 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:38.267524 8346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:38.267784 8346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:38.267827 8346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:38.271725 8346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:38.271783 8346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:38.271802 8346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:38.271819 8346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:38.271835 8346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:38.277172 8346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34553
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:38.278185 8346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:38.279161 8346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:38.287773 8352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:38.287779 8351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:38.287796 8354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:38.288285 8346 server_base.cc:1034] running on GCE node
I20260809 07:52:38.288851 8346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:38.289978 8346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:38.291150 8346 hybrid_clock.cc:648] HybridClock initialized: now 1786261958291122 us; error 40 us; skew 500 ppm
Aug 09 07:52:38 dist-test-slave-10pc krb5kdc[8330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261958, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:38.293929 8346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:38.294912 8346 webserver.cc:460] Webserver started at http://127.0.0.1:44573/ using document root <none> and password file <none>
I20260809 07:52:38.295536 8346 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:38.295630 8346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:38.295861 8346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:38.298027 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "9706049e95be48e7ab7384fe0237e7a6"
format_stamp: "Formatted at 2026-08-09 07:52:38 on dist-test-slave-10pc"
server_key: "5329d781d58c01c60aa4b5c08f85c64b"
server_key_iv: "9ce19d82fa59b9968484ceb19baec968"
server_key_version: "encryptionkey@0"
I20260809 07:52:38.298628 8346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "9706049e95be48e7ab7384fe0237e7a6"
format_stamp: "Formatted at 2026-08-09 07:52:38 on dist-test-slave-10pc"
server_key: "5329d781d58c01c60aa4b5c08f85c64b"
server_key_iv: "9ce19d82fa59b9968484ceb19baec968"
server_key_version: "encryptionkey@0"
I20260809 07:52:38.302533 8346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:38.305195 8361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:38.306605 8346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:38.306766 8346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "9706049e95be48e7ab7384fe0237e7a6"
format_stamp: "Formatted at 2026-08-09 07:52:38 on dist-test-slave-10pc"
server_key: "5329d781d58c01c60aa4b5c08f85c64b"
server_key_iv: "9ce19d82fa59b9968484ceb19baec968"
server_key_version: "encryptionkey@0"
I20260809 07:52:38.306895 8346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:38.311786 8346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:38.315022 8346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:38.315219 8346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:38.322662 8346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34553
I20260809 07:52:38.322690 8417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34553 every 8 connection(s)
I20260809 07:52:38.323640 8346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:38.326555 8418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:38.327735 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8346
I20260809 07:52:38.327884 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:38.328249 18717 external_mini_cluster.cc:1428] Setting key 7903fdabffa62bec208e9feaa5afec61
I20260809 07:52:38.333336 8418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6: Bootstrap starting.
I20260809 07:52:38.333824 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:38.329831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34629 (local address 127.0.0.1:34553)
0809 07:52:38.330296 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:38.330311 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:38.330346 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:52:38.331002 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:38.331019 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:38.331321 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:38.331678 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:38.332553 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:38.332557 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:38.332570 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:38.332582 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:38.332640 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:38.332660 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:38.332808 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:38.332884 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 07:52:38.333049 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":149,"threads_started":1}
I20260809 07:52:38.335763 8418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:38.336643 8418 log.cc:824] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:38.336687 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:38.338750 8418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6: No bootstrap required, opened a new log
I20260809 07:52:38.342684 8418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9706049e95be48e7ab7384fe0237e7a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260809 07:52:38.342926 8418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:38.342988 8418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9706049e95be48e7ab7384fe0237e7a6, State: Initialized, Role: FOLLOWER
I20260809 07:52:38.343433 8418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9706049e95be48e7ab7384fe0237e7a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260809 07:52:38.343559 8418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:38.343679 8418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:38.343806 8418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:38.344859 8418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9706049e95be48e7ab7384fe0237e7a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260809 07:52:38.345389 8418 leader_election.cc:302] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9706049e95be48e7ab7384fe0237e7a6; no voters:
I20260809 07:52:38.345911 8418 leader_election.cc:288] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:38.346089 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:38.338875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47641 (local address 127.0.0.1:34553)
0809 07:52:38.338983 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:38.338988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:38.339205 (+ 217us) server_negotiation.cc:407] Connection header received
0809 07:52:38.339327 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:38.339329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:38.339357 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:38.339421 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:38.345508 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:38.345511 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:38.345513 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:38.345527 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:38.345574 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:38.345617 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:38.345730 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:38.345898 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 07:52:38.345948 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 07:52:38.346148 8428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:38.346395 8428 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [term 1 LEADER]: Becoming Leader. State: Replica: 9706049e95be48e7ab7384fe0237e7a6, State: Running, Role: LEADER
I20260809 07:52:38.346760 8428 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9706049e95be48e7ab7384fe0237e7a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } }
I20260809 07:52:38.347422 8418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:38.348651 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9706049e95be48e7ab7384fe0237e7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9706049e95be48e7ab7384fe0237e7a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } } }
I20260809 07:52:38.348793 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:38.348930 8431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9706049e95be48e7ab7384fe0237e7a6. Latest consensus state: current_term: 1 leader_uuid: "9706049e95be48e7ab7384fe0237e7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9706049e95be48e7ab7384fe0237e7a6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34553 } } }
I20260809 07:52:38.349097 8431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:38.349224 8439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:38.352262 8439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:38.357640 8439 catalog_manager.cc:1347] Generated new cluster ID: 6638c0b2bd0e4953b8ce2492e2905de0
I20260809 07:52:38.357710 8439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:38.365957 8439 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:38.366290 8439 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:38.370894 8439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9706049e95be48e7ab7384fe0237e7a6: Generated new TSK 0
I20260809 07:52:38.371726 8439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:38.384142 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8346
2026-08-09T07:52:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:38 dist-test-slave-10pc krb5kdc[8451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:38 dist-test-slave-10pc krb5kdc[8451](info): set up 2 sockets
Aug 09 07:52:38 dist-test-slave-10pc krb5kdc[8451](info): commencing operation
krb5kdc: starting...
W20260809 07:52:39.463459 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:39 dist-test-slave-10pc krb5kdc[8451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:39.592022 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:39.704679 8466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:39.704983 8466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:39.705067 8466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:39.709033 8466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:39.709116 8466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:39.709159 8466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:39.709223 8466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:39.709263 8466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:39.714425 8466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:39.715513 8466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:39.716584 8466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:39.724769 8475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:39.724819 8472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:39.724782 8473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:39.725101 8466 server_base.cc:1034] running on GCE node
I20260809 07:52:39.725611 8466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:39.726689 8466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:39.727852 8466 hybrid_clock.cc:648] HybridClock initialized: now 1786261959727795 us; error 67 us; skew 500 ppm
Aug 09 07:52:39 dist-test-slave-10pc krb5kdc[8451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261959, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:39.730710 8466 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:39.731541 8466 webserver.cc:460] Webserver started at http://127.0.0.1:36861/ using document root <none> and password file <none>
I20260809 07:52:39.732112 8466 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:39.732173 8466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:39.732369 8466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:39.734670 8466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "2257f92176d741cc8c38186a5e9e7d74"
format_stamp: "Formatted at 2026-08-09 07:52:39 on dist-test-slave-10pc"
server_key: "a4b000c2ffac0aaf050df1b3fb55f3a1"
server_key_iv: "e03900f166ae995341418c1e833202a0"
server_key_version: "encryptionkey@0"
I20260809 07:52:39.735208 8466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "2257f92176d741cc8c38186a5e9e7d74"
format_stamp: "Formatted at 2026-08-09 07:52:39 on dist-test-slave-10pc"
server_key: "a4b000c2ffac0aaf050df1b3fb55f3a1"
server_key_iv: "e03900f166ae995341418c1e833202a0"
server_key_version: "encryptionkey@0"
I20260809 07:52:39.738919 8466 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:52:39.741449 8483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:39.742697 8466 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:39.742800 8466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "2257f92176d741cc8c38186a5e9e7d74"
format_stamp: "Formatted at 2026-08-09 07:52:39 on dist-test-slave-10pc"
server_key: "a4b000c2ffac0aaf050df1b3fb55f3a1"
server_key_iv: "e03900f166ae995341418c1e833202a0"
server_key_version: "encryptionkey@0"
I20260809 07:52:39.742888 8466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:39.756598 8466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:39.759289 8466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:39.759436 8466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:39.766800 8466 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34359
I20260809 07:52:39.766829 8538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34359 every 8 connection(s)
I20260809 07:52:39.767735 8466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:39.768689 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8466
I20260809 07:52:39.768831 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:39.769136 18717 external_mini_cluster.cc:1428] Setting key 8e9a2ae8d58620852f27db99d17fd98b
I20260809 07:52:39.770891 8539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:39.776188 8539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74: Bootstrap starting.
Aug 09 07:52:39 dist-test-slave-10pc krb5kdc[8451](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 07:52:39.778735 8539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:39.779425 8539 log.cc:824] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:39.781338 8539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74: No bootstrap required, opened a new log
I20260809 07:52:39.783172 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:39.770482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35167 (local address 127.0.0.1:34359)
0809 07:52:39.770901 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:39.770916 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:39.770950 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:52:39.771631 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:39.771646 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:39.771964 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:39.772305 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:39.773149 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:39.774128 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:39.774985 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:39.775455 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:39.775627 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:39.778422 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:39.778427 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:39.778439 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:39.778464 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:39.780415 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:39.780908 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:39.780916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:39.780926 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:39.781031 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:39.781338 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:39.781342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:39.781344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:39.781762 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:39.781958 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:39.782244 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 07:52:39.782517 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":134,"threads_started":1}
I20260809 07:52:39.785151 8539 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2257f92176d741cc8c38186a5e9e7d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260809 07:52:39.785444 8539 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:39.785526 8539 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2257f92176d741cc8c38186a5e9e7d74, State: Initialized, Role: FOLLOWER
I20260809 07:52:39.786077 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:39.786026 8539 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2257f92176d741cc8c38186a5e9e7d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260809 07:52:39.786183 8539 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:39.786245 8539 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:39.786334 8539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:39.787577 8539 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2257f92176d741cc8c38186a5e9e7d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260809 07:52:39.787936 8539 leader_election.cc:302] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2257f92176d741cc8c38186a5e9e7d74; no voters:
I20260809 07:52:39.788275 8539 leader_election.cc:288] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:39.788383 8549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:39.788743 8549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [term 1 LEADER]: Becoming Leader. State: Replica: 2257f92176d741cc8c38186a5e9e7d74, State: Running, Role: LEADER
I20260809 07:52:39.789134 8549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2257f92176d741cc8c38186a5e9e7d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } }
I20260809 07:52:39.789508 8539 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:39.790443 8551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2257f92176d741cc8c38186a5e9e7d74. Latest consensus state: current_term: 1 leader_uuid: "2257f92176d741cc8c38186a5e9e7d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2257f92176d741cc8c38186a5e9e7d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } } }
I20260809 07:52:39.790572 8551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:39.791246 8552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2257f92176d741cc8c38186a5e9e7d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2257f92176d741cc8c38186a5e9e7d74" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34359 } } }
I20260809 07:52:39.791368 8552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:39.792201 8559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:39.795712 8559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:39.797334 8543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:39.788445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42345 (local address 127.0.0.1:34359)
0809 07:52:39.788563 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:39.788569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:39.788787 (+ 218us) server_negotiation.cc:407] Connection header received
0809 07:52:39.788968 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:39.788971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:39.789012 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:39.789058 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:39.789923 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:39.790714 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:39.791951 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:39.792258 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:39.792338 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:39.793873 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:39.793876 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:39.793879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:39.793902 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:39.795550 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:39.796021 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:39.796024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:39.796026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:39.796125 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:39.796436 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:39.796439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:39.796441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:39.796669 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:39.796792 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:39.797060 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 07:52:39.797185 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":24}
I20260809 07:52:39.802017 8559 catalog_manager.cc:1347] Generated new cluster ID: 0819034d434e409d8df6ee7e266181bd
I20260809 07:52:39.802079 8559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:39.819602 8559 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:39.820856 8559 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:39.829485 8559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2257f92176d741cc8c38186a5e9e7d74: Generated new TSK 0
I20260809 07:52:39.830302 8559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:39.841542 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8466
2026-08-09T07:52:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1457 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:39 dist-test-slave-10pc krb5kdc[8572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:39 dist-test-slave-10pc krb5kdc[8572](info): set up 2 sockets
Aug 09 07:52:39 dist-test-slave-10pc krb5kdc[8572](info): commencing operation
krb5kdc: starting...
W20260809 07:52:40.922830 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:41 dist-test-slave-10pc krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:41.051817 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35185
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:41.163237 8586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:41.163569 8586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:41.163653 8586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:41.167802 8586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:41.167914 8586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:41.167954 8586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:41.168015 8586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:41.168052 8586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:41.173360 8586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:41.174510 8586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:41.175591 8586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:41.184604 8592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:41.184623 8591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:41.184612 8594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:41.184672 8586 server_base.cc:1034] running on GCE node
I20260809 07:52:41.185417 8586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:41.186546 8586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:41.187748 8586 hybrid_clock.cc:648] HybridClock initialized: now 1786261961187736 us; error 27 us; skew 500 ppm
Aug 09 07:52:41 dist-test-slave-10pc krb5kdc[8572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261961, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:41.190749 8586 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:41.191605 8586 webserver.cc:460] Webserver started at http://127.0.0.1:36137/ using document root <none> and password file <none>
I20260809 07:52:41.192124 8586 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:41.192183 8586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:41.192348 8586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:41.194427 8586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "f30a56de7938460b9b4a293bcf9067df"
format_stamp: "Formatted at 2026-08-09 07:52:41 on dist-test-slave-10pc"
server_key: "7d882b12505d4f83ad107519989314e7"
server_key_iv: "564678698e09b925df7df57fcd8d642b"
server_key_version: "encryptionkey@0"
I20260809 07:52:41.194962 8586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "f30a56de7938460b9b4a293bcf9067df"
format_stamp: "Formatted at 2026-08-09 07:52:41 on dist-test-slave-10pc"
server_key: "7d882b12505d4f83ad107519989314e7"
server_key_iv: "564678698e09b925df7df57fcd8d642b"
server_key_version: "encryptionkey@0"
I20260809 07:52:41.198586 8586 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:52:41.200897 8601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:41.202157 8586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:41.202309 8586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "f30a56de7938460b9b4a293bcf9067df"
format_stamp: "Formatted at 2026-08-09 07:52:41 on dist-test-slave-10pc"
server_key: "7d882b12505d4f83ad107519989314e7"
server_key_iv: "564678698e09b925df7df57fcd8d642b"
server_key_version: "encryptionkey@0"
I20260809 07:52:41.202421 8586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:41.222121 8586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:41.225139 8586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:41.225360 8586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:41.232355 8586 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35185
I20260809 07:52:41.232383 8656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35185 every 8 connection(s)
I20260809 07:52:41.233368 8586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:41.236202 8657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:41.238641 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8586
I20260809 07:52:41.238798 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:41.239127 18717 external_mini_cluster.cc:1428] Setting key 57a201387a7765a9873a5f33b2b93ecd
I20260809 07:52:41.242377 8657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df: Bootstrap starting.
I20260809 07:52:41.245195 8657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:41.246112 8657 log.cc:824] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:41.248428 8657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df: No bootstrap required, opened a new log
I20260809 07:52:41.249341 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:41.240607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50097 (local address 127.0.0.1:35185)
0809 07:52:41.241110 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:41.241133 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:41.241169 (+ 36us) server_negotiation.cc:407] Connection header received
0809 07:52:41.241808 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:41.241825 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:41.242181 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:41.242523 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:41.243003 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:41.244412 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:41.245539 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:41.246259 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:41.246428 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:41.247360 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:41.247367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:41.247382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:41.247865 (+ 483us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:41.247931 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:41.247960 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:41.248053 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:41.248208 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 07:52:41.248411 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":144,"threads_started":1}
I20260809 07:52:41.252143 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:41.252758 8657 raft_consensus.cc:348] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30a56de7938460b9b4a293bcf9067df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260809 07:52:41.253000 8657 raft_consensus.cc:374] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:41.253059 8657 raft_consensus.cc:729] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f30a56de7938460b9b4a293bcf9067df, State: Initialized, Role: FOLLOWER
I20260809 07:52:41.253511 8657 consensus_queue.cc:260] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30a56de7938460b9b4a293bcf9067df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260809 07:52:41.253677 8657 raft_consensus.cc:388] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:41.253768 8657 raft_consensus.cc:482] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:41.253868 8657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:41.255133 8657 raft_consensus.cc:504] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30a56de7938460b9b4a293bcf9067df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260809 07:52:41.255507 8657 leader_election.cc:302] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f30a56de7938460b9b4a293bcf9067df; no voters:
I20260809 07:52:41.255838 8657 leader_election.cc:288] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:41.255957 8668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:41.256250 8668 raft_consensus.cc:686] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [term 1 LEADER]: Becoming Leader. State: Replica: f30a56de7938460b9b4a293bcf9067df, State: Running, Role: LEADER
I20260809 07:52:41.256598 8668 consensus_queue.cc:237] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30a56de7938460b9b4a293bcf9067df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } }
I20260809 07:52:41.257067 8657 sys_catalog.cc:564] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:41.258347 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f30a56de7938460b9b4a293bcf9067df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30a56de7938460b9b4a293bcf9067df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } } }
I20260809 07:52:41.258491 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:41.258867 8670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [sys.catalog]: SysCatalogTable state changed. Reason: New leader f30a56de7938460b9b4a293bcf9067df. Latest consensus state: current_term: 1 leader_uuid: "f30a56de7938460b9b4a293bcf9067df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f30a56de7938460b9b4a293bcf9067df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35185 } } }
I20260809 07:52:41.258926 8677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:41.259326 8660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:41.254359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48051 (local address 127.0.0.1:35185)
0809 07:52:41.254464 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:41.254469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:41.254758 (+ 289us) server_negotiation.cc:407] Connection header received
0809 07:52:41.254829 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:41.254832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:41.254878 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:41.254965 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:41.255255 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:41.256004 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:41.257203 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:41.257580 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:41.257685 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:41.258588 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:41.258592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:41.258594 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:41.258619 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:41.258644 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:41.258654 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:41.258763 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:41.259105 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 07:52:41.259168 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":30}
I20260809 07:52:41.260860 8670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:41.262372 8677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:41.268411 8677 catalog_manager.cc:1347] Generated new cluster ID: 5b9d7d12a8854557b1e2a2e2f57e69fc
I20260809 07:52:41.268510 8677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:41.275935 8677 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:41.277190 8677 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:41.285157 8677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f30a56de7938460b9b4a293bcf9067df: Generated new TSK 0
I20260809 07:52:41.285815 8677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:41.296330 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8586
2026-08-09T07:52:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:41 dist-test-slave-10pc krb5kdc[8689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:41 dist-test-slave-10pc krb5kdc[8689](info): set up 2 sockets
Aug 09 07:52:41 dist-test-slave-10pc krb5kdc[8689](info): commencing operation
krb5kdc: starting...
W20260809 07:52:42.451726 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:42 dist-test-slave-10pc krb5kdc[8689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:42.583928 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:42.697599 8702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:42.697911 8702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:42.697996 8702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:42.701979 8702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:42.702075 8702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:42.702114 8702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:42.702152 8702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:42.702189 8702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:42.707362 8702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:42.708388 8702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:42.709436 8702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:42.717988 8709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:42.717988 8711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:42.718011 8708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:42.718357 8702 server_base.cc:1034] running on GCE node
I20260809 07:52:42.718894 8702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:42.720013 8702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:42.721206 8702 hybrid_clock.cc:648] HybridClock initialized: now 1786261962721159 us; error 59 us; skew 500 ppm
Aug 09 07:52:42 dist-test-slave-10pc krb5kdc[8689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261962, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:42.724046 8702 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:42.725013 8702 webserver.cc:460] Webserver started at http://127.0.0.1:45775/ using document root <none> and password file <none>
I20260809 07:52:42.725592 8702 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:42.725696 8702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:42.725932 8702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:42.727993 8702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "d59c3df593204f1ca3e0a56cd5c61fe2"
format_stamp: "Formatted at 2026-08-09 07:52:42 on dist-test-slave-10pc"
server_key: "30fd548c1757f40db92e32a6a3e51202"
server_key_iv: "d71d1404d81c390eb6f4e554c5b51948"
server_key_version: "encryptionkey@0"
I20260809 07:52:42.728668 8702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "d59c3df593204f1ca3e0a56cd5c61fe2"
format_stamp: "Formatted at 2026-08-09 07:52:42 on dist-test-slave-10pc"
server_key: "30fd548c1757f40db92e32a6a3e51202"
server_key_iv: "d71d1404d81c390eb6f4e554c5b51948"
server_key_version: "encryptionkey@0"
I20260809 07:52:42.732553 8702 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:52:42.734956 8719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:42.736168 8702 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:42.736307 8702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "d59c3df593204f1ca3e0a56cd5c61fe2"
format_stamp: "Formatted at 2026-08-09 07:52:42 on dist-test-slave-10pc"
server_key: "30fd548c1757f40db92e32a6a3e51202"
server_key_iv: "d71d1404d81c390eb6f4e554c5b51948"
server_key_version: "encryptionkey@0"
I20260809 07:52:42.736436 8702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:42.741089 8702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:42.743947 8702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:42.744143 8702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:42.751320 8702 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41599
I20260809 07:52:42.751387 8773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41599 every 8 connection(s)
I20260809 07:52:42.752241 8702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:42.755010 8774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:42.760399 8774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2: Bootstrap starting.
I20260809 07:52:42.761140 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8702
I20260809 07:52:42.761299 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:42.761588 18717 external_mini_cluster.cc:1428] Setting key 1ad77ea63d7dde279304188c89cf3828
I20260809 07:52:42.763191 8774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:42.763928 8774 log.cc:824] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:42.766207 8774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2: No bootstrap required, opened a new log
I20260809 07:52:42.766997 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:42.762794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50877 (local address 127.0.0.1:41599)
0809 07:52:42.763285 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:42.763323 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:42.763366 (+ 43us) server_negotiation.cc:407] Connection header received
0809 07:52:42.764006 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:42.764023 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:42.764401 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:42.764814 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:42.765676 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:42.765680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:42.765691 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:42.765702 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:42.765757 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:42.765775 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:42.765863 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:42.766043 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 07:52:42.766258 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":158,"threads_started":1}
I20260809 07:52:42.770000 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:42.770304 8774 raft_consensus.cc:348] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260809 07:52:42.770543 8774 raft_consensus.cc:374] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:42.770619 8774 raft_consensus.cc:729] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d59c3df593204f1ca3e0a56cd5c61fe2, State: Initialized, Role: FOLLOWER
I20260809 07:52:42.771068 8774 consensus_queue.cc:260] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260809 07:52:42.771221 8774 raft_consensus.cc:388] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:42.771293 8774 raft_consensus.cc:482] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:42.771420 8774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:42.772411 8774 raft_consensus.cc:504] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260809 07:52:42.772804 8774 leader_election.cc:302] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d59c3df593204f1ca3e0a56cd5c61fe2; no voters:
I20260809 07:52:42.773147 8774 leader_election.cc:288] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:42.773245 8785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:42.773504 8785 raft_consensus.cc:686] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [term 1 LEADER]: Becoming Leader. State: Replica: d59c3df593204f1ca3e0a56cd5c61fe2, State: Running, Role: LEADER
I20260809 07:52:42.773867 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:42.772119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47777 (local address 127.0.0.1:41599)
0809 07:52:42.772239 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:42.772243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:42.772390 (+ 147us) server_negotiation.cc:407] Connection header received
0809 07:52:42.772550 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:42.772553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:42.772584 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:42.772666 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:42.773421 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:42.773428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:42.773434 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:42.773450 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:42.773478 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:42.773490 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:42.773575 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:42.773726 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 07:52:42.773762 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 07:52:42.773842 8785 consensus_queue.cc:237] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } }
I20260809 07:52:42.774552 8774 sys_catalog.cc:564] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:42.775153 8785 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d59c3df593204f1ca3e0a56cd5c61fe2. Latest consensus state: current_term: 1 leader_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } } }
I20260809 07:52:42.775278 8785 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:42.775610 8786 sys_catalog.cc:455] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d59c3df593204f1ca3e0a56cd5c61fe2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41599 } } }
I20260809 07:52:42.775770 8786 sys_catalog.cc:458] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:42.775769 8792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:42.778788 8792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:42.784155 8792 catalog_manager.cc:1347] Generated new cluster ID: 756dc19aa8ae462d9b9b0738fb20731e
I20260809 07:52:42.784229 8792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:42.796504 8792 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:42.796773 8792 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:42.803939 8792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d59c3df593204f1ca3e0a56cd5c61fe2: Generated new TSK 0
I20260809 07:52:42.804613 8792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:42.813006 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8702
2026-08-09T07:52:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:42 dist-test-slave-10pc krb5kdc[8807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:42 dist-test-slave-10pc krb5kdc[8807](info): set up 2 sockets
Aug 09 07:52:42 dist-test-slave-10pc krb5kdc[8807](info): commencing operation
krb5kdc: starting...
W20260809 07:52:43.887980 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:43 dist-test-slave-10pc krb5kdc[8807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:44.015735 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36137
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36137
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:44.127430 8820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:44.127657 8820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:44.127693 8820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:44.131526 8820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:44.131573 8820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:44.131592 8820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:44.131608 8820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:44.131623 8820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:44.136729 8820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:44.137641 8820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:44.138546 8820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:44.147030 8826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:44.146973 8828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:44.147336 8820 server_base.cc:1034] running on GCE node
W20260809 07:52:44.146968 8825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:44.147897 8820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:44.149056 8820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:44.150247 8820 hybrid_clock.cc:648] HybridClock initialized: now 1786261964150217 us; error 42 us; skew 500 ppm
Aug 09 07:52:44 dist-test-slave-10pc krb5kdc[8807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261964, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:44.157588 8820 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:44.158515 8820 webserver.cc:460] Webserver started at http://127.0.0.1:34637/ using document root <none> and password file <none>
I20260809 07:52:44.159078 8820 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:44.159169 8820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:44.159407 8820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:44.161553 8820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "ba0edff0029a4db6a70e99889505a44f"
format_stamp: "Formatted at 2026-08-09 07:52:44 on dist-test-slave-10pc"
server_key: "3bc8bd1e7d37bd5328ea4db2acfaeca2"
server_key_iv: "5d9b2bab53787221b5f45dd1fa67dea9"
server_key_version: "encryptionkey@0"
I20260809 07:52:44.162128 8820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "ba0edff0029a4db6a70e99889505a44f"
format_stamp: "Formatted at 2026-08-09 07:52:44 on dist-test-slave-10pc"
server_key: "3bc8bd1e7d37bd5328ea4db2acfaeca2"
server_key_iv: "5d9b2bab53787221b5f45dd1fa67dea9"
server_key_version: "encryptionkey@0"
I20260809 07:52:44.165851 8820 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 07:52:44.168285 8835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:44.169631 8820 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:44.169797 8820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "ba0edff0029a4db6a70e99889505a44f"
format_stamp: "Formatted at 2026-08-09 07:52:44 on dist-test-slave-10pc"
server_key: "3bc8bd1e7d37bd5328ea4db2acfaeca2"
server_key_iv: "5d9b2bab53787221b5f45dd1fa67dea9"
server_key_version: "encryptionkey@0"
I20260809 07:52:44.169914 8820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:44.190423 8820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:44.193408 8820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:44.193626 8820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:44.201148 8820 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36137
I20260809 07:52:44.201176 8890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36137 every 8 connection(s)
I20260809 07:52:44.202118 8820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:44.202778 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8820
I20260809 07:52:44.202932 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:44.203270 18717 external_mini_cluster.cc:1428] Setting key 11e29734571d977902c0679886d0c688
I20260809 07:52:44.205518 8891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:44.211102 8891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f: Bootstrap starting.
Aug 09 07:52:44 dist-test-slave-10pc krb5kdc[8807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 07:52:44.213712 8891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:44.214561 8891 log.cc:824] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:44.216785 8891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f: No bootstrap required, opened a new log
I20260809 07:52:44.217866 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:44.204745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:52339 (local address 127.0.0.1:36137)
0809 07:52:44.205221 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:44.205237 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:44.205274 (+ 37us) server_negotiation.cc:407] Connection header received
0809 07:52:44.205982 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:44.206002 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:44.206355 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:44.206695 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:44.207547 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:44.208633 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:44.209521 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:44.209997 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:44.210175 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:44.213068 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:44.213073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:44.213086 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:44.213119 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:44.214891 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:44.215474 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:44.215479 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:44.215484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:44.215589 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:44.215934 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:44.215936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:44.215937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:44.216314 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:44.216528 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:44.216875 (+ 347us) server_negotiation.cc:299] Negotiation successful
0809 07:52:44.217180 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":166,"threads_started":1}
I20260809 07:52:44.220337 8891 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0edff0029a4db6a70e99889505a44f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36137 } }
I20260809 07:52:44.220613 8891 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:44.220688 8891 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0edff0029a4db6a70e99889505a44f, State: Initialized, Role: FOLLOWER
I20260809 07:52:44.220849 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:44.221146 8891 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0edff0029a4db6a70e99889505a44f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36137 } }
I20260809 07:52:44.221294 8891 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:44.221367 8891 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:44.221469 8891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:44.222618 8891 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0edff0029a4db6a70e99889505a44f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36137 } }
I20260809 07:52:44.222997 8891 leader_election.cc:302] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba0edff0029a4db6a70e99889505a44f; no voters:
I20260809 07:52:44.223349 8891 leader_election.cc:288] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:44.223446 8903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:44.223663 8903 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [term 1 LEADER]: Becoming Leader. State: Replica: ba0edff0029a4db6a70e99889505a44f, State: Running, Role: LEADER
I20260809 07:52:44.224040 8903 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0edff0029a4db6a70e99889505a44f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36137 } }
I20260809 07:52:44.224699 8891 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:44.226097 8905 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba0edff0029a4db6a70e99889505a44f. Latest consensus state: current_term: 1 leader_uuid: "ba0edff0029a4db6a70e99889505a44f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0edff0029a4db6a70e99889505a44f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36137 } } }
I20260809 07:52:44.226130 8904 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba0edff0029a4db6a70e99889505a44f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba0edff0029a4db6a70e99889505a44f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36137 } } }
I20260809 07:52:44.226223 8905 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:44.226244 8904 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:44.229940 8918 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:44.230026 8918 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:44.230095 8914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:44.231086 8914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:44.232851 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:44.223097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:59027 (local address 127.0.0.1:36137)
0809 07:52:44.223245 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:44.223249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:44.223468 (+ 219us) server_negotiation.cc:407] Connection header received
0809 07:52:44.223648 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:44.223652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:44.223702 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:44.223780 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:44.224710 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:44.225502 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:44.226705 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:44.227039 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:44.227146 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:44.228925 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:44.228928 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:44.228931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:44.228954 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:44.230594 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:44.231199 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:44.231203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:44.231205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:44.231283 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:44.231682 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:44.231686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:44.231687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:44.231887 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:44.232021 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:44.232615 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 07:52:44.232707 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 07:52:44.236860 8914 catalog_manager.cc:1347] Generated new cluster ID: bc3900684fba4823897c52c444b56a08
I20260809 07:52:44.236918 8914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:44.269769 8914 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:44.270805 8914 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:44.279008 8914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba0edff0029a4db6a70e99889505a44f: Generated new TSK 0
I20260809 07:52:44.279737 8914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:44.291728 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8820
W20260809 07:52:44.300885 8805 connection.cc:535] client connection to 127.0.0.1:36137 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36137): Connection reset by peer (error 104)
2026-08-09T07:52:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:44 dist-test-slave-10pc krb5kdc[8924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:44 dist-test-slave-10pc krb5kdc[8924](info): set up 2 sockets
Aug 09 07:52:44 dist-test-slave-10pc krb5kdc[8924](info): commencing operation
krb5kdc: starting...
W20260809 07:52:45.368678 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:45 dist-test-slave-10pc krb5kdc[8924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:45.497422 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45775
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:45.612041 8938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:45.612303 8938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:45.612344 8938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:45.616477 8938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:45.616531 8938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:45.616550 8938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:45.616567 8938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:45.616582 8938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:45.621600 8938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:45.622526 8938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:45.623479 8938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:45.632128 8947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:45.632172 8943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:45.632133 8944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:45.632519 8938 server_base.cc:1034] running on GCE node
I20260809 07:52:45.633229 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:45.634330 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:45.635528 8938 hybrid_clock.cc:648] HybridClock initialized: now 1786261965635501 us; error 38 us; skew 500 ppm
Aug 09 07:52:45 dist-test-slave-10pc krb5kdc[8924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261965, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:45.638305 8938 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:45.639134 8938 webserver.cc:460] Webserver started at http://127.0.0.1:39477/ using document root <none> and password file <none>
I20260809 07:52:45.639648 8938 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:45.639708 8938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:45.639868 8938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:45.641940 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "8dbd2a6e27e740df951669259f430f01"
format_stamp: "Formatted at 2026-08-09 07:52:45 on dist-test-slave-10pc"
server_key: "a239d4d7d11192c9db33a88069c0675b"
server_key_iv: "1193616753e72c151550be39c08e0df4"
server_key_version: "encryptionkey@0"
I20260809 07:52:45.642457 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "8dbd2a6e27e740df951669259f430f01"
format_stamp: "Formatted at 2026-08-09 07:52:45 on dist-test-slave-10pc"
server_key: "a239d4d7d11192c9db33a88069c0675b"
server_key_iv: "1193616753e72c151550be39c08e0df4"
server_key_version: "encryptionkey@0"
I20260809 07:52:45.646173 8938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:52:45.648512 8954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:45.649709 8938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:45.649834 8938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "8dbd2a6e27e740df951669259f430f01"
format_stamp: "Formatted at 2026-08-09 07:52:45 on dist-test-slave-10pc"
server_key: "a239d4d7d11192c9db33a88069c0675b"
server_key_iv: "1193616753e72c151550be39c08e0df4"
server_key_version: "encryptionkey@0"
I20260809 07:52:45.649938 8938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:45.675704 8938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:45.678639 8938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:45.678853 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:45.686707 8938 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45775
I20260809 07:52:45.686733 9008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45775 every 8 connection(s)
I20260809 07:52:45.687711 8938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:45.690507 9009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:45.694298 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 8938
I20260809 07:52:45.694449 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:45.694710 18717 external_mini_cluster.cc:1428] Setting key 8813fefdfb3bb8e3f11982aa43ea4d71
I20260809 07:52:45.696524 9009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01: Bootstrap starting.
I20260809 07:52:45.698781 9009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:45.699522 9009 log.cc:824] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:45.701505 9009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01: No bootstrap required, opened a new log
I20260809 07:52:45.703907 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:45.696128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:57241 (local address 127.0.0.1:45775)
0809 07:52:45.696624 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:45.696659 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:45.696705 (+ 46us) server_negotiation.cc:407] Connection header received
0809 07:52:45.697351 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:45.697368 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:45.697710 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:45.698049 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:45.698484 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:45.699578 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:45.700717 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:45.701192 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:45.701369 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:45.702301 (+ 932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:45.702306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:45.702319 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:45.702691 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:45.702741 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:45.702763 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:45.702901 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:45.703088 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 07:52:45.703224 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":134,"threads_started":1}
I20260809 07:52:45.705438 9009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dbd2a6e27e740df951669259f430f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260809 07:52:45.705718 9009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:45.705809 9009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dbd2a6e27e740df951669259f430f01, State: Initialized, Role: FOLLOWER
I20260809 07:52:45.706287 9009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dbd2a6e27e740df951669259f430f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260809 07:52:45.706447 9009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:45.706537 9009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:45.706664 9009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:45.706991 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:45.707772 9009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dbd2a6e27e740df951669259f430f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260809 07:52:45.708134 9009 leader_election.cc:302] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dbd2a6e27e740df951669259f430f01; no voters:
I20260809 07:52:45.708535 9009 leader_election.cc:288] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:45.708621 9017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:45.708904 9017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [term 1 LEADER]: Becoming Leader. State: Replica: 8dbd2a6e27e740df951669259f430f01, State: Running, Role: LEADER
I20260809 07:52:45.709434 9017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dbd2a6e27e740df951669259f430f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } }
I20260809 07:52:45.709918 9009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:45.710541 9017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dbd2a6e27e740df951669259f430f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dbd2a6e27e740df951669259f430f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } } }
I20260809 07:52:45.710721 9017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:45.711055 9019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dbd2a6e27e740df951669259f430f01. Latest consensus state: current_term: 1 leader_uuid: "8dbd2a6e27e740df951669259f430f01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dbd2a6e27e740df951669259f430f01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45775 } } }
I20260809 07:52:45.711140 9030 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:45.711242 9019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:45.714514 9030 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:45.716965 9012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:45.709281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:53347 (local address 127.0.0.1:45775)
0809 07:52:45.709396 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:45.709401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:45.712421 (+ 3020us) server_negotiation.cc:407] Connection header received
0809 07:52:45.712643 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:45.712646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:45.712695 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:45.712779 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:45.713093 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:45.713815 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:45.715012 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:45.715332 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:45.715452 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:45.716433 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:45.716438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:45.716441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:45.716475 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:45.716504 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:45.716519 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:45.716610 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:45.716805 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 07:52:45.716841 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 07:52:45.721045 9030 catalog_manager.cc:1347] Generated new cluster ID: 745318e6298a489da2d7f24e6358c0f8
I20260809 07:52:45.721115 9030 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:45.726768 9030 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:45.727661 9030 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:45.735139 9030 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dbd2a6e27e740df951669259f430f01: Generated new TSK 0
I20260809 07:52:45.735771 9030 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:45.744222 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 8938
W20260809 07:52:45.752007 8922 connection.cc:535] client connection to 127.0.0.1:45775 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45775): Connection reset by peer (error 104)
2026-08-09T07:52:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:45 dist-test-slave-10pc krb5kdc[9043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:45 dist-test-slave-10pc krb5kdc[9043](info): set up 2 sockets
Aug 09 07:52:45 dist-test-slave-10pc krb5kdc[9043](info): commencing operation
krb5kdc: starting...
W20260809 07:52:46.835997 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:46 dist-test-slave-10pc krb5kdc[9043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:46.963172 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38651
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:47.076736 9057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:47.077052 9057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:47.077133 9057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:47.081122 9057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:47.081211 9057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:47.081251 9057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:47.081310 9057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:47.081351 9057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:47.086580 9057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:47.087812 9057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:47.088943 9057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:47.098131 9064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:47.098253 9057 server_base.cc:1034] running on GCE node
W20260809 07:52:47.098165 9063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:47.098423 9066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:47.099014 9057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:47.100050 9057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:47.101254 9057 hybrid_clock.cc:648] HybridClock initialized: now 1786261967101233 us; error 31 us; skew 500 ppm
Aug 09 07:52:47 dist-test-slave-10pc krb5kdc[9043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261967, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:47.104164 9057 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:47.105129 9057 webserver.cc:460] Webserver started at http://127.0.0.1:43649/ using document root <none> and password file <none>
I20260809 07:52:47.105654 9057 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:47.105713 9057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:47.105877 9057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:47.107865 9057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "7b536fa5e18c4c5e88561e87beb675a0"
format_stamp: "Formatted at 2026-08-09 07:52:47 on dist-test-slave-10pc"
server_key: "6b588f72b63a773ae29bca95613d487e"
server_key_iv: "0a8beb2a5ca43246d17013b6b33b5c61"
server_key_version: "encryptionkey@0"
I20260809 07:52:47.108376 9057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "7b536fa5e18c4c5e88561e87beb675a0"
format_stamp: "Formatted at 2026-08-09 07:52:47 on dist-test-slave-10pc"
server_key: "6b588f72b63a773ae29bca95613d487e"
server_key_iv: "0a8beb2a5ca43246d17013b6b33b5c61"
server_key_version: "encryptionkey@0"
I20260809 07:52:47.112608 9057 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 07:52:47.114951 9074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:47.116395 9057 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:47.116572 9057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "7b536fa5e18c4c5e88561e87beb675a0"
format_stamp: "Formatted at 2026-08-09 07:52:47 on dist-test-slave-10pc"
server_key: "6b588f72b63a773ae29bca95613d487e"
server_key_iv: "0a8beb2a5ca43246d17013b6b33b5c61"
server_key_version: "encryptionkey@0"
I20260809 07:52:47.116693 9057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:47.121606 9057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:47.124637 9057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:47.124828 9057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:47.131796 9057 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38651
I20260809 07:52:47.131825 9128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38651 every 8 connection(s)
I20260809 07:52:47.132778 9057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:47.135417 9129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:47.139405 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9057
I20260809 07:52:47.139555 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:47.139832 18717 external_mini_cluster.cc:1428] Setting key 4172a5589c105d10c8b1e0bf4b176254
I20260809 07:52:47.141145 9129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0: Bootstrap starting.
I20260809 07:52:47.143447 9129 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:47.144250 9129 log.cc:824] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:47.145287 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:47.141252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:36041 (local address 127.0.0.1:38651)
0809 07:52:47.141680 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:47.141694 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:47.141729 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:52:47.142378 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:47.142395 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:47.142680 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:47.143014 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:47.143840 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:47.143844 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:47.143856 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:47.143880 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:47.143933 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:47.143952 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:47.144062 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:47.144169 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 07:52:47.144355 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":118,"threads_started":1}
I20260809 07:52:47.146311 9129 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0: No bootstrap required, opened a new log
I20260809 07:52:47.148065 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:47.150156 9129 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b536fa5e18c4c5e88561e87beb675a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38651 } }
I20260809 07:52:47.150451 9129 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:47.150537 9129 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b536fa5e18c4c5e88561e87beb675a0, State: Initialized, Role: FOLLOWER
I20260809 07:52:47.151032 9129 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b536fa5e18c4c5e88561e87beb675a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38651 } }
I20260809 07:52:47.151234 9129 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:47.151319 9129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:47.151428 9129 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:47.151969 9133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:47.150097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:51727 (local address 127.0.0.1:38651)
0809 07:52:47.150226 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:47.150231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:47.150407 (+ 176us) server_negotiation.cc:407] Connection header received
0809 07:52:47.150547 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:47.150551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:47.150596 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:47.150677 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:47.151470 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:47.151474 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:47.151478 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:47.151493 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:47.151521 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:47.151535 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:47.151618 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:47.151761 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 07:52:47.151809 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 07:52:47.152606 9129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b536fa5e18c4c5e88561e87beb675a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38651 } }
I20260809 07:52:47.152973 9129 leader_election.cc:302] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b536fa5e18c4c5e88561e87beb675a0; no voters:
I20260809 07:52:47.153314 9129 leader_election.cc:288] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:47.153421 9141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:47.153654 9141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [term 1 LEADER]: Becoming Leader. State: Replica: 7b536fa5e18c4c5e88561e87beb675a0, State: Running, Role: LEADER
I20260809 07:52:47.153996 9141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b536fa5e18c4c5e88561e87beb675a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38651 } }
I20260809 07:52:47.154506 9129 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:47.155646 9142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b536fa5e18c4c5e88561e87beb675a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b536fa5e18c4c5e88561e87beb675a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38651 } } }
I20260809 07:52:47.155802 9142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:47.156332 9150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:47.156373 9143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b536fa5e18c4c5e88561e87beb675a0. Latest consensus state: current_term: 1 leader_uuid: "7b536fa5e18c4c5e88561e87beb675a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b536fa5e18c4c5e88561e87beb675a0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38651 } } }
I20260809 07:52:47.156503 9143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:47.159148 9150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:47.164194 9150 catalog_manager.cc:1347] Generated new cluster ID: 4c3bb81879384ef3a4942bb3cb066288
I20260809 07:52:47.164260 9150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:47.191749 9150 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:47.192152 9150 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:47.197722 9150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b536fa5e18c4c5e88561e87beb675a0: Generated new TSK 0
I20260809 07:52:47.198370 9150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:47.204005 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9057
2026-08-09T07:52:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1461 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:47 dist-test-slave-10pc krb5kdc[9162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:47 dist-test-slave-10pc krb5kdc[9162](info): set up 2 sockets
Aug 09 07:52:47 dist-test-slave-10pc krb5kdc[9162](info): commencing operation
krb5kdc: starting...
W20260809 07:52:48.283207 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:48 dist-test-slave-10pc krb5kdc[9162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:48.413194 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:48.527742 9176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:48.528040 9176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:48.528084 9176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:48.532152 9176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:48.532219 9176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:48.532239 9176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:48.532255 9176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:48.532271 9176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:48.537541 9176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44555
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:48.538548 9176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:48.539592 9176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:48.548121 9181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:48.548134 9184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:48.548121 9182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:48.548492 9176 server_base.cc:1034] running on GCE node
I20260809 07:52:48.549077 9176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:48.550189 9176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:48.551378 9176 hybrid_clock.cc:648] HybridClock initialized: now 1786261968551347 us; error 43 us; skew 500 ppm
Aug 09 07:52:48 dist-test-slave-10pc krb5kdc[9162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261968, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:48.554330 9176 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:48.555243 9176 webserver.cc:460] Webserver started at http://127.0.0.1:34053/ using document root <none> and password file <none>
I20260809 07:52:48.555787 9176 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:48.555850 9176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:48.556021 9176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:48.558127 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "d6bf9afa617c41e088ffd225f57ce420"
format_stamp: "Formatted at 2026-08-09 07:52:48 on dist-test-slave-10pc"
server_key: "406ca404ee0b01704bd088c84a8b363a"
server_key_iv: "b94326f79e4005829aa9d1a6443741b6"
server_key_version: "encryptionkey@0"
I20260809 07:52:48.558660 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "d6bf9afa617c41e088ffd225f57ce420"
format_stamp: "Formatted at 2026-08-09 07:52:48 on dist-test-slave-10pc"
server_key: "406ca404ee0b01704bd088c84a8b363a"
server_key_iv: "b94326f79e4005829aa9d1a6443741b6"
server_key_version: "encryptionkey@0"
I20260809 07:52:48.562492 9176 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:52:48.564870 9192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:48.566172 9176 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:48.566313 9176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "d6bf9afa617c41e088ffd225f57ce420"
format_stamp: "Formatted at 2026-08-09 07:52:48 on dist-test-slave-10pc"
server_key: "406ca404ee0b01704bd088c84a8b363a"
server_key_iv: "b94326f79e4005829aa9d1a6443741b6"
server_key_version: "encryptionkey@0"
I20260809 07:52:48.566422 9176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:48.580878 9176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:48.584028 9176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:48.584252 9176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:48.591616 9176 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44555
I20260809 07:52:48.591647 9249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44555 every 8 connection(s)
I20260809 07:52:48.592658 9176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:48.595554 9250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:48.600251 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9176
I20260809 07:52:48.600389 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:48.600669 18717 external_mini_cluster.cc:1428] Setting key 6a468e2ec4212b5a61faa2e260a11c10
I20260809 07:52:48.601490 9250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420: Bootstrap starting.
I20260809 07:52:48.603899 9250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:48.604724 9250 log.cc:824] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:48.606642 9250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420: No bootstrap required, opened a new log
Aug 09 07:52:48 dist-test-slave-10pc krb5kdc[9162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 07:52:48.610646 9250 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bf9afa617c41e088ffd225f57ce420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260809 07:52:48.610939 9250 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:48.611011 9250 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6bf9afa617c41e088ffd225f57ce420, State: Initialized, Role: FOLLOWER
I20260809 07:52:48.611577 9250 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bf9afa617c41e088ffd225f57ce420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260809 07:52:48.611753 9250 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:48.611825 9250 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:48.611927 9250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:48.613081 9250 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bf9afa617c41e088ffd225f57ce420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260809 07:52:48.613456 9250 leader_election.cc:302] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6bf9afa617c41e088ffd225f57ce420; no voters:
I20260809 07:52:48.613834 9250 leader_election.cc:288] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:48.613927 9256 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:48.614192 9256 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [term 1 LEADER]: Becoming Leader. State: Replica: d6bf9afa617c41e088ffd225f57ce420, State: Running, Role: LEADER
I20260809 07:52:48.614537 9256 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bf9afa617c41e088ffd225f57ce420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } }
I20260809 07:52:48.615233 9250 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:48.616269 9258 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6bf9afa617c41e088ffd225f57ce420. Latest consensus state: current_term: 1 leader_uuid: "d6bf9afa617c41e088ffd225f57ce420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bf9afa617c41e088ffd225f57ce420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } } }
I20260809 07:52:48.616379 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:48.602091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:60981 (local address 127.0.0.1:44555)
0809 07:52:48.602525 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:48.602541 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:48.602580 (+ 39us) server_negotiation.cc:407] Connection header received
0809 07:52:48.603199 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:48.603216 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:48.603563 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:48.603897 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:48.604764 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:48.605772 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:48.606801 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:48.607511 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:48.607708 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:48.610275 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:48.610280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:48.610296 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:48.610327 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:48.612552 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:48.613065 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:48.613072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:48.613078 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:48.613228 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:48.613560 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:48.613564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:48.613565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:48.614113 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:48.614904 (+ 791us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:48.615098 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 07:52:48.615334 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":132,"threads_started":1}
I20260809 07:52:48.616405 9258 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:48.616604 9257 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6bf9afa617c41e088ffd225f57ce420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6bf9afa617c41e088ffd225f57ce420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44555 } } }
I20260809 07:52:48.616681 9257 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:48.619855 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 07:52:48.620348 9274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:48.620434 9274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:48.620530 9269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:48.621503 9269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:48.627683 9269 catalog_manager.cc:1347] Generated new cluster ID: b5bcb46734a84b3e847c300dd3aeb1c7
I20260809 07:52:48.627741 9269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:48.631323 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:48.622424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:37271 (local address 127.0.0.1:44555)
0809 07:52:48.622522 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:48.622526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:48.622539 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:52:48.622576 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:48.622578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:48.622620 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:48.622715 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:48.623497 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:48.624005 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:48.624866 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:48.625113 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:48.625214 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:48.627225 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:48.627229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:48.627231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:48.627258 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:48.629576 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:48.630068 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:48.630071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:48.630072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:48.630145 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:48.630468 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:48.630471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:48.630473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:48.630637 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:48.630789 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:48.631063 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 07:52:48.631151 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 07:52:48.637231 9269 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:48.638401 9269 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:48.645370 9269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6bf9afa617c41e088ffd225f57ce420: Generated new TSK 0
I20260809 07:52:48.646104 9269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:48.650362 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9176
W20260809 07:52:48.659521 9160 connection.cc:535] client connection to 127.0.0.1:44555 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44555): Connection reset by peer (error 104)
2026-08-09T07:52:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:48 dist-test-slave-10pc krb5kdc[9285](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:48 dist-test-slave-10pc krb5kdc[9285](info): set up 2 sockets
Aug 09 07:52:48 dist-test-slave-10pc krb5kdc[9285](info): commencing operation
krb5kdc: starting...
W20260809 07:52:49.741633 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:49 dist-test-slave-10pc krb5kdc[9285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:49.871963 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32773
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:49.983769 9299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:49.984028 9299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:49.984069 9299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:49.987939 9299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:49.987998 9299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:49.988018 9299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:49.988034 9299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:49.988049 9299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:49.993152 9299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32773
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:49.994150 9299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:49.995101 9299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:50.003671 9304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:50.003753 9299 server_base.cc:1034] running on GCE node
W20260809 07:52:50.003770 9305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:50.003671 9307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:50.004583 9299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:50.005749 9299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:50.006939 9299 hybrid_clock.cc:648] HybridClock initialized: now 1786261970006918 us; error 35 us; skew 500 ppm
Aug 09 07:52:50 dist-test-slave-10pc krb5kdc[9285](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261970, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:50.015455 9299 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:50.016444 9299 webserver.cc:460] Webserver started at http://127.0.0.1:45297/ using document root <none> and password file <none>
I20260809 07:52:50.017113 9299 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:50.017181 9299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:50.017421 9299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:50.019557 9299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "5019d27ed85d4c17ae5996a44c2ef9e4"
format_stamp: "Formatted at 2026-08-09 07:52:50 on dist-test-slave-10pc"
server_key: "52650cb4e5b369d3a0747c21e0377172"
server_key_iv: "c45e3a902b5e53ced77e6c8434923aa6"
server_key_version: "encryptionkey@0"
I20260809 07:52:50.020149 9299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "5019d27ed85d4c17ae5996a44c2ef9e4"
format_stamp: "Formatted at 2026-08-09 07:52:50 on dist-test-slave-10pc"
server_key: "52650cb4e5b369d3a0747c21e0377172"
server_key_iv: "c45e3a902b5e53ced77e6c8434923aa6"
server_key_version: "encryptionkey@0"
I20260809 07:52:50.023936 9299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 07:52:50.026405 9314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:50.027627 9299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:50.027779 9299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "5019d27ed85d4c17ae5996a44c2ef9e4"
format_stamp: "Formatted at 2026-08-09 07:52:50 on dist-test-slave-10pc"
server_key: "52650cb4e5b369d3a0747c21e0377172"
server_key_iv: "c45e3a902b5e53ced77e6c8434923aa6"
server_key_version: "encryptionkey@0"
I20260809 07:52:50.027880 9299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:50.057852 9299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:50.061169 9299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:50.061340 9299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:50.068593 9299 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32773
I20260809 07:52:50.068619 9372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32773 every 8 connection(s)
I20260809 07:52:50.069592 9299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:50.072499 9373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:50.077957 9373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4: Bootstrap starting.
I20260809 07:52:50.078591 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9299
I20260809 07:52:50.078751 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:50.079013 18717 external_mini_cluster.cc:1428] Setting key 784f269ecf9943f98a5e560bca1d5b58
I20260809 07:52:50.081043 9373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:50.081792 9373 log.cc:824] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:50.083601 9373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4: No bootstrap required, opened a new log
I20260809 07:52:50.087164 9373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260809 07:52:50.087400 9373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:50.087493 9373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5019d27ed85d4c17ae5996a44c2ef9e4, State: Initialized, Role: FOLLOWER
I20260809 07:52:50.087968 9373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260809 07:52:50.088126 9373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:50.088209 9373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:50.088308 9373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:50.088574 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:50.080724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:40237 (local address 127.0.0.1:32773)
0809 07:52:50.081192 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:50.081208 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:50.081255 (+ 47us) server_negotiation.cc:407] Connection header received
0809 07:52:50.081947 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:50.081966 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:50.082302 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:50.082636 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:50.083056 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.084111 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:50.085085 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.085577 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:50.085761 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:50.086694 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:50.086699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:50.086712 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:50.087094 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:50.087147 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:50.087170 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:50.087308 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:50.087493 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 07:52:50.087651 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":130,"threads_started":1}
I20260809 07:52:50.089430 9373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260809 07:52:50.089777 9373 leader_election.cc:302] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5019d27ed85d4c17ae5996a44c2ef9e4; no voters:
I20260809 07:52:50.090106 9373 leader_election.cc:288] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:50.090164 9380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:50.090420 9380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [term 1 LEADER]: Becoming Leader. State: Replica: 5019d27ed85d4c17ae5996a44c2ef9e4, State: Running, Role: LEADER
I20260809 07:52:50.090786 9380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } }
I20260809 07:52:50.091372 9373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:50.091756 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:50.092000 9380 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5019d27ed85d4c17ae5996a44c2ef9e4. Latest consensus state: current_term: 1 leader_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } } }
I20260809 07:52:50.092134 9380 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:50.094393 9382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5019d27ed85d4c17ae5996a44c2ef9e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32773 } } }
I20260809 07:52:50.094502 9382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4 [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:50.096187 9400 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:50.096305 9400 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:50.096387 9387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:50.097393 9387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:50.101588 9376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:50.096708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:57285 (local address 127.0.0.1:32773)
0809 07:52:50.096810 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:50.096815 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:50.096990 (+ 175us) server_negotiation.cc:407] Connection header received
0809 07:52:50.097235 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:50.097238 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:50.097288 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:50.097379 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:50.097687 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.098453 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:50.099664 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:50.099973 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:50.100099 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:50.101051 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:50.101054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:50.101057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:50.101069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:50.101097 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:50.101114 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:50.101184 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:50.101388 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 07:52:50.101421 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 07:52:50.103150 9387 catalog_manager.cc:1347] Generated new cluster ID: 75d8c264aa994e2ca6289601bf828967
I20260809 07:52:50.103220 9387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:50.117117 9387 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:50.118060 9387 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:50.123754 9387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5019d27ed85d4c17ae5996a44c2ef9e4: Generated new TSK 0
I20260809 07:52:50.124413 9387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:50.130331 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9299
W20260809 07:52:50.139642 9283 connection.cc:535] client connection to 127.0.0.1:32773 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32773): Connection reset by peer (error 104)
2026-08-09T07:52:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:50 dist-test-slave-10pc krb5kdc[9407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:50 dist-test-slave-10pc krb5kdc[9407](info): set up 2 sockets
Aug 09 07:52:50 dist-test-slave-10pc krb5kdc[9407](info): commencing operation
krb5kdc: starting...
W20260809 07:52:51.220312 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:51 dist-test-slave-10pc krb5kdc[9407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261971, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 07:52:51.350940 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34655
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:51.463559 9421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:51.463853 9421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:51.463928 9421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:51.467945 9421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:51.468042 9421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:51.468080 9421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:51.468123 9421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:51.468160 9421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:51.473349 9421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34655
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:51.474444 9421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:51.475479 9421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:51.484145 9426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:51.484193 9427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:51.484229 9429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:51.484505 9421 server_base.cc:1034] running on GCE node
I20260809 07:52:51.485062 9421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:51.486166 9421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:51.487322 9421 hybrid_clock.cc:648] HybridClock initialized: now 1786261971487310 us; error 26 us; skew 500 ppm
Aug 09 07:52:51 dist-test-slave-10pc krb5kdc[9407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261971, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:51.490115 9421 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 07:52:51.490985 9421 webserver.cc:460] Webserver started at http://127.0.0.1:34779/ using document root <none> and password file <none>
I20260809 07:52:51.491567 9421 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:51.491662 9421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:51.491891 9421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:51.493973 9421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "10260c803d4543cc91364749aad309f4"
format_stamp: "Formatted at 2026-08-09 07:52:51 on dist-test-slave-10pc"
server_key: "2ca7bab38d6df0ea60ac0bcf3a342aef"
server_key_iv: "c094577026b873bd9c22634c544796e2"
server_key_version: "encryptionkey@0"
I20260809 07:52:51.494539 9421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "10260c803d4543cc91364749aad309f4"
format_stamp: "Formatted at 2026-08-09 07:52:51 on dist-test-slave-10pc"
server_key: "2ca7bab38d6df0ea60ac0bcf3a342aef"
server_key_iv: "c094577026b873bd9c22634c544796e2"
server_key_version: "encryptionkey@0"
I20260809 07:52:51.498250 9421 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:52:51.500675 9436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:51.501850 9421 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:51.501986 9421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "10260c803d4543cc91364749aad309f4"
format_stamp: "Formatted at 2026-08-09 07:52:51 on dist-test-slave-10pc"
server_key: "2ca7bab38d6df0ea60ac0bcf3a342aef"
server_key_iv: "c094577026b873bd9c22634c544796e2"
server_key_version: "encryptionkey@0"
I20260809 07:52:51.502094 9421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:51.506703 9421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:51.509972 9421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:51.510171 9421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:51.517266 9421 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34655
I20260809 07:52:51.517295 9493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34655 every 8 connection(s)
I20260809 07:52:51.518219 9421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:51.521137 9494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:51.526726 9494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4: Bootstrap starting.
I20260809 07:52:51.527853 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9421
I20260809 07:52:51.527981 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:51.528307 18717 external_mini_cluster.cc:1428] Setting key 068d9099a747dac04a8621e5101e00c5
I20260809 07:52:51.529639 9494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:51.530589 9494 log.cc:824] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:51.532881 9494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4: No bootstrap required, opened a new log
I20260809 07:52:51.534144 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:51.529518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:45899 (local address 127.0.0.1:34655)
0809 07:52:51.530082 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:51.530102 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:51.530145 (+ 43us) server_negotiation.cc:407] Connection header received
0809 07:52:51.531064 (+ 919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:51.531095 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:51.531425 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:51.531792 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:51.532592 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:51.532598 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:51.532612 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:51.532625 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:51.532697 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:51.532725 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:51.532838 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:51.532919 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 07:52:51.533247 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":176,"threads_started":1}
I20260809 07:52:51.536568 9494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10260c803d4543cc91364749aad309f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 07:52:51.536813 9494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:51.536885 9494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10260c803d4543cc91364749aad309f4, State: Initialized, Role: FOLLOWER
I20260809 07:52:51.537364 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:51.537375 9494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10260c803d4543cc91364749aad309f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 07:52:51.537547 9494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:51.537614 9494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:51.537714 9494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:51.538606 9494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10260c803d4543cc91364749aad309f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 07:52:51.539011 9494 leader_election.cc:302] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10260c803d4543cc91364749aad309f4; no voters:
I20260809 07:52:51.539302 9494 leader_election.cc:288] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:51.539377 9505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:51.539659 9505 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [term 1 LEADER]: Becoming Leader. State: Replica: 10260c803d4543cc91364749aad309f4, State: Running, Role: LEADER
I20260809 07:52:51.539950 9505 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10260c803d4543cc91364749aad309f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } }
I20260809 07:52:51.540555 9494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:51.541110 9507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10260c803d4543cc91364749aad309f4. Latest consensus state: current_term: 1 leader_uuid: "10260c803d4543cc91364749aad309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10260c803d4543cc91364749aad309f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } } }
I20260809 07:52:51.541244 9507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:51.541898 9506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10260c803d4543cc91364749aad309f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10260c803d4543cc91364749aad309f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34655 } } }
I20260809 07:52:51.541954 9512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:51.542038 9506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:51.542382 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:51.539518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.141:35311 (local address 127.0.0.1:34655)
0809 07:52:51.539650 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:51.539654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:51.540725 (+ 1071us) server_negotiation.cc:407] Connection header received
0809 07:52:51.540868 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:51.540871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:51.540914 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:51.540990 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:51.541900 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:51.541903 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:51.541906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:51.541917 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:51.541948 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:51.541963 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:51.542037 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:51.542230 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 07:52:51.542270 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:52:51.545384 9512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:51.551198 9512 catalog_manager.cc:1347] Generated new cluster ID: 3f7fb76ddedd433e985ce8cb3042695a
I20260809 07:52:51.551378 9512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:51.561578 9512 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:51.561939 9512 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:51.565994 9512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10260c803d4543cc91364749aad309f4: Generated new TSK 0
I20260809 07:52:51.566658 9512 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:51.572152 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9421
2026-08-09T07:52:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:51 dist-test-slave-10pc krb5kdc[9529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:51 dist-test-slave-10pc krb5kdc[9529](info): set up 2 sockets
Aug 09 07:52:51 dist-test-slave-10pc krb5kdc[9529](info): commencing operation
krb5kdc: starting...
W20260809 07:52:52.668140 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:52 dist-test-slave-10pc krb5kdc[9529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:52.795513 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40959
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:44091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:40959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:52.906505 9543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:52.906757 9543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:52.906797 9543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:52.910620 9543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:52.910673 9543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:52.910693 9543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:52.910709 9543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:52.910724 9543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:52.915961 9543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:44091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:40959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:52.916934 9543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:52.917899 9543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 07:52:52.926402 9543 server_base.cc:1034] running on GCE node
W20260809 07:52:52.926355 9553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:52.926376 9548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:52.926344 9550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:52.927232 9543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:52.928334 9543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:52.929529 9543 hybrid_clock.cc:648] HybridClock initialized: now 1786261972929513 us; error 26 us; skew 500 ppm
Aug 09 07:52:52 dist-test-slave-10pc krb5kdc[9529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261972, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:52.932384 9543 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:52.933276 9543 webserver.cc:460] Webserver started at http://127.18.71.126:35489/ using document root <none> and password file <none>
I20260809 07:52:52.933817 9543 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:52.933878 9543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:52.934132 9543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:52.936203 9543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "4ac81ec86eff4c4492c110e197de46a0"
format_stamp: "Formatted at 2026-08-09 07:52:52 on dist-test-slave-10pc"
server_key: "f0d936ea20ca3340706719cebd6b3a75"
server_key_iv: "54ab3ac8f53bbec5a2e7be9c2fa035ea"
server_key_version: "encryptionkey@0"
I20260809 07:52:52.936816 9543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "4ac81ec86eff4c4492c110e197de46a0"
format_stamp: "Formatted at 2026-08-09 07:52:52 on dist-test-slave-10pc"
server_key: "f0d936ea20ca3340706719cebd6b3a75"
server_key_iv: "54ab3ac8f53bbec5a2e7be9c2fa035ea"
server_key_version: "encryptionkey@0"
I20260809 07:52:52.940394 9543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:52.942804 9560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:52.943966 9543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:52:52.944097 9543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "4ac81ec86eff4c4492c110e197de46a0"
format_stamp: "Formatted at 2026-08-09 07:52:52 on dist-test-slave-10pc"
server_key: "f0d936ea20ca3340706719cebd6b3a75"
server_key_iv: "54ab3ac8f53bbec5a2e7be9c2fa035ea"
server_key_version: "encryptionkey@0"
I20260809 07:52:52.944195 9543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:52.960850 9543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:52.963757 9543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:52.963953 9543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:52.970863 9543 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:40959
I20260809 07:52:52.970903 9616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:40959 every 8 connection(s)
I20260809 07:52:52.971817 9543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:52.974699 9617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:52.980156 9617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0: Bootstrap starting.
I20260809 07:52:52.982041 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9543
I20260809 07:52:52.982178 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:52.982453 18717 external_mini_cluster.cc:1428] Setting key daf31cc00ae0196a5a4d33e49741105f
I20260809 07:52:52.982667 9617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:52.983556 9617 log.cc:824] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:52.985498 9617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0: No bootstrap required, opened a new log
I20260809 07:52:52.988900 9617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac81ec86eff4c4492c110e197de46a0" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40959 } }
I20260809 07:52:52.989143 9617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:52.989225 9617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ac81ec86eff4c4492c110e197de46a0, State: Initialized, Role: FOLLOWER
I20260809 07:52:52.989701 9617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac81ec86eff4c4492c110e197de46a0" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40959 } }
I20260809 07:52:52.989856 9617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:52.989943 9617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:52.990061 9617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:52.991005 9617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac81ec86eff4c4492c110e197de46a0" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40959 } }
Aug 09 07:52:52 dist-test-slave-10pc krb5kdc[9529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:52.991379 9617 leader_election.cc:302] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ac81ec86eff4c4492c110e197de46a0; no voters:
I20260809 07:52:52.991735 9617 leader_election.cc:288] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:52.991851 9622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:52.992100 9622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [term 1 LEADER]: Becoming Leader. State: Replica: 4ac81ec86eff4c4492c110e197de46a0, State: Running, Role: LEADER
I20260809 07:52:52.992535 9622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac81ec86eff4c4492c110e197de46a0" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40959 } }
I20260809 07:52:52.993148 9617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:52.994539 9624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ac81ec86eff4c4492c110e197de46a0. Latest consensus state: current_term: 1 leader_uuid: "4ac81ec86eff4c4492c110e197de46a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac81ec86eff4c4492c110e197de46a0" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40959 } } }
I20260809 07:52:52.994678 9624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:52.995997 9623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ac81ec86eff4c4492c110e197de46a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ac81ec86eff4c4492c110e197de46a0" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40959 } } }
I20260809 07:52:52.996155 9623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0 [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:52.998677 9638 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:52.998801 9638 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:52.998929 9633 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:52.999665 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:52.983540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34589 (local address 127.18.71.126:40959)
0809 07:52:52.983994 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:52.984023 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:52.984069 (+ 46us) server_negotiation.cc:407] Connection header received
0809 07:52:52.984854 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:52.984874 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:52.985221 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:52.985558 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:52.986403 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:52.987389 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:52.988272 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:52.988836 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:52.989020 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:52.992217 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:52.992224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:52.992242 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:52.992274 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:52.994963 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:52.995659 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:52.995665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:52.995670 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:52.995809 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:52.996306 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:52.996309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:52.996311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:52.997635 (+ 1324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:52.997830 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:52.998533 (+ 703us) server_negotiation.cc:299] Negotiation successful
0809 07:52:52.998748 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":131,"threads_started":1}
I20260809 07:52:52.999862 9633 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:53.003501 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:53.006326 9633 catalog_manager.cc:1347] Generated new cluster ID: fb25810ac562429697aa308fd458529c
I20260809 07:52:53.006388 9633 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:53.013718 9633 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:53.014804 9633 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:53.017583 9620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:53.005640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37963 (local address 127.18.71.126:40959)
0809 07:52:53.005767 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:53.005772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:53.005986 (+ 214us) server_negotiation.cc:407] Connection header received
0809 07:52:53.006119 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:53.006122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:53.006171 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:53.006266 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:53.009274 (+ 3008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:53.010076 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:53.011232 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:53.011590 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:53.011697 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:53.013783 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:53.013787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:53.013790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:53.013815 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:53.015726 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:53.016261 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:53.016265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:53.016267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:53.016356 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:53.016711 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:53.016715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:53.016717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:53.016912 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:53.017064 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:53.017313 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 07:52:53.017411 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":49}
I20260809 07:52:53.021415 9633 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ac81ec86eff4c4492c110e197de46a0: Generated new TSK 0
I20260809 07:52:53.022209 9633 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:53.029476 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9543
W20260809 07:52:53.038796 9526 connection.cc:535] client connection to 127.18.71.126:40959 recv error: Network error: failed to read from TLS socket (remote: 127.18.71.126:40959): Connection reset by peer (error 104)
2026-08-09T07:52:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:53 dist-test-slave-10pc krb5kdc[9652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:53 dist-test-slave-10pc krb5kdc[9652](info): set up 2 sockets
Aug 09 07:52:53 dist-test-slave-10pc krb5kdc[9652](info): commencing operation
krb5kdc: starting...
W20260809 07:52:54.121515 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:54 dist-test-slave-10pc krb5kdc[9652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:54.251646 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:41467
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:38563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:41467
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:54.367660 9665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:54.367993 9665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:54.368069 9665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:54.372189 9665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:54.372300 9665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:54.372341 9665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:54.372390 9665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:54.372422 9665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:54.378276 9665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:38563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:41467
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:41467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:54.379477 9665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:54.380595 9665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:54.389356 9670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:54.389389 9671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:54.389356 9674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:54.389643 9665 server_base.cc:1034] running on GCE node
I20260809 07:52:54.390270 9665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:54.391427 9665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:54.392638 9665 hybrid_clock.cc:648] HybridClock initialized: now 1786261974392617 us; error 31 us; skew 500 ppm
Aug 09 07:52:54 dist-test-slave-10pc krb5kdc[9652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261974, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:54.395467 9665 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:54.396409 9665 webserver.cc:460] Webserver started at http://127.18.71.126:42847/ using document root <none> and password file <none>
I20260809 07:52:54.397010 9665 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:54.397106 9665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:54.397339 9665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:54.399374 9665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "ba482704cc8740839156ed9b71780176"
format_stamp: "Formatted at 2026-08-09 07:52:54 on dist-test-slave-10pc"
server_key: "69334de4790d8f7c00fe507476ed727c"
server_key_iv: "52f3c150caff4d51d38884630e223def"
server_key_version: "encryptionkey@0"
I20260809 07:52:54.399941 9665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "ba482704cc8740839156ed9b71780176"
format_stamp: "Formatted at 2026-08-09 07:52:54 on dist-test-slave-10pc"
server_key: "69334de4790d8f7c00fe507476ed727c"
server_key_iv: "52f3c150caff4d51d38884630e223def"
server_key_version: "encryptionkey@0"
I20260809 07:52:54.403756 9665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 07:52:54.406252 9681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:54.407461 9665 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:54.407604 9665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "ba482704cc8740839156ed9b71780176"
format_stamp: "Formatted at 2026-08-09 07:52:54 on dist-test-slave-10pc"
server_key: "69334de4790d8f7c00fe507476ed727c"
server_key_iv: "52f3c150caff4d51d38884630e223def"
server_key_version: "encryptionkey@0"
I20260809 07:52:54.407713 9665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:54.422760 9665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:54.425623 9665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:54.425832 9665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:54.432965 9665 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:41467
I20260809 07:52:54.432993 9737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:41467 every 8 connection(s)
I20260809 07:52:54.433916 9665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:54.436721 9738 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:54.438848 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9665
I20260809 07:52:54.438992 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:54.439263 18717 external_mini_cluster.cc:1428] Setting key 431967ce5327a5562ad47a5e5cc75856
I20260809 07:52:54.443070 9738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176: Bootstrap starting.
I20260809 07:52:54.445346 9738 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:54.446110 9738 log.cc:824] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:54.448675 9738 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176: No bootstrap required, opened a new log
I20260809 07:52:54.449108 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:54.440784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40511 (local address 127.18.71.126:41467)
0809 07:52:54.441258 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:54.441274 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:54.441312 (+ 38us) server_negotiation.cc:407] Connection header received
0809 07:52:54.442163 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:54.442184 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:54.442531 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:54.442886 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:54.443345 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:54.444309 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:54.445255 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:54.445792 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:54.445978 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:54.447026 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:54.447031 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:54.447044 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:54.447553 (+ 509us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:54.447621 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:54.447651 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:54.447793 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:54.447996 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 07:52:54.448132 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":136,"threads_started":1}
I20260809 07:52:54.452104 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:54.452906 9738 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba482704cc8740839156ed9b71780176" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41467 } }
I20260809 07:52:54.453159 9738 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:54.453236 9738 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba482704cc8740839156ed9b71780176, State: Initialized, Role: FOLLOWER
I20260809 07:52:54.453711 9738 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba482704cc8740839156ed9b71780176" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41467 } }
I20260809 07:52:54.453866 9738 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:54.453956 9738 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:54.454078 9738 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:54.455340 9738 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba482704cc8740839156ed9b71780176" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41467 } }
I20260809 07:52:54.455719 9738 leader_election.cc:302] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba482704cc8740839156ed9b71780176; no voters:
I20260809 07:52:54.456060 9738 leader_election.cc:288] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:54.456171 9749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:54.456434 9749 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [term 1 LEADER]: Becoming Leader. State: Replica: ba482704cc8740839156ed9b71780176, State: Running, Role: LEADER
I20260809 07:52:54.456833 9749 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba482704cc8740839156ed9b71780176" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41467 } }
I20260809 07:52:54.457438 9738 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:54.458595 9751 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba482704cc8740839156ed9b71780176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba482704cc8740839156ed9b71780176" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41467 } } }
I20260809 07:52:54.458880 9742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:54.454158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50601 (local address 127.18.71.126:41467)
0809 07:52:54.454294 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:54.454298 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:54.454544 (+ 246us) server_negotiation.cc:407] Connection header received
0809 07:52:54.454659 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:54.454662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:54.454698 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:54.454764 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:54.455063 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:54.455618 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:54.456830 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:54.457174 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:54.457279 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:54.458205 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:54.458212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:54.458217 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:54.458232 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:54.458262 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:54.458281 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:54.458359 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:54.458655 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 07:52:54.458690 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 07:52:54.458786 9751 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:54.460033 9749 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba482704cc8740839156ed9b71780176. Latest consensus state: current_term: 1 leader_uuid: "ba482704cc8740839156ed9b71780176" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba482704cc8740839156ed9b71780176" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 41467 } } }
I20260809 07:52:54.460189 9749 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176 [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:54.462366 9767 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:54.462462 9767 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:54.462539 9764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:54.463232 9764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:54.468362 9764 catalog_manager.cc:1347] Generated new cluster ID: e7c32faf846e4dcda596cfad4737c588
I20260809 07:52:54.468431 9764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:54.478609 9764 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:54.479817 9764 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:54.483965 9764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba482704cc8740839156ed9b71780176: Generated new TSK 0
I20260809 07:52:54.484766 9764 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:54.494751 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9665
W20260809 07:52:54.502332 9649 connection.cc:535] client connection to 127.18.71.126:41467 recv error: Network error: failed to read from TLS socket (remote: 127.18.71.126:41467): Connection reset by peer (error 104)
2026-08-09T07:52:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1463 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:54 dist-test-slave-10pc krb5kdc[9773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:54 dist-test-slave-10pc krb5kdc[9773](info): set up 2 sockets
Aug 09 07:52:54 dist-test-slave-10pc krb5kdc[9773](info): commencing operation
krb5kdc: starting...
W20260809 07:52:55.572774 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:55 dist-test-slave-10pc krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:55.700716 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40523
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:43463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:40523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:55.812672 9787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:55.812916 9787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:55.812956 9787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:55.816900 9787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:55.816954 9787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:55.816975 9787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:55.816991 9787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:55.817008 9787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:55.822211 9787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:43463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:40523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:40523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:55.823144 9787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:55.824100 9787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:55.832531 9792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:55.832541 9793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:55.832542 9795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:55.832595 9787 server_base.cc:1034] running on GCE node
I20260809 07:52:55.833381 9787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:55.834445 9787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:55.835603 9787 hybrid_clock.cc:648] HybridClock initialized: now 1786261975835570 us; error 45 us; skew 500 ppm
Aug 09 07:52:55 dist-test-slave-10pc krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261975, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:55.843003 9787 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:55.843931 9787 webserver.cc:460] Webserver started at http://127.18.71.126:39067/ using document root <none> and password file <none>
I20260809 07:52:55.844561 9787 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:55.844653 9787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:55.844885 9787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:55.846935 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "d34ad40a6dbc4e16a06b6c3649c38147"
format_stamp: "Formatted at 2026-08-09 07:52:55 on dist-test-slave-10pc"
server_key: "719406a1dab8365f757c349e06c98c34"
server_key_iv: "dc96cb8bba224cfc0c948964a02ff20d"
server_key_version: "encryptionkey@0"
I20260809 07:52:55.847491 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "d34ad40a6dbc4e16a06b6c3649c38147"
format_stamp: "Formatted at 2026-08-09 07:52:55 on dist-test-slave-10pc"
server_key: "719406a1dab8365f757c349e06c98c34"
server_key_iv: "dc96cb8bba224cfc0c948964a02ff20d"
server_key_version: "encryptionkey@0"
I20260809 07:52:55.851140 9787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 07:52:55.853504 9802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:55.854691 9787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:52:55.854930 9787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "d34ad40a6dbc4e16a06b6c3649c38147"
format_stamp: "Formatted at 2026-08-09 07:52:55 on dist-test-slave-10pc"
server_key: "719406a1dab8365f757c349e06c98c34"
server_key_iv: "dc96cb8bba224cfc0c948964a02ff20d"
server_key_version: "encryptionkey@0"
I20260809 07:52:55.855046 9787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:55.859582 9787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:55.862428 9787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:55.862627 9787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:55.870198 9787 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:40523
I20260809 07:52:55.870227 9858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:40523 every 8 connection(s)
I20260809 07:52:55.871124 9787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:55.873821 9859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:55.877163 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9787
I20260809 07:52:55.877315 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:55.877588 18717 external_mini_cluster.cc:1428] Setting key 5bbe2c8bf0921c755f561eb42ce3a61e
I20260809 07:52:55.879516 9859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147: Bootstrap starting.
I20260809 07:52:55.881784 9859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:55.882534 9859 log.cc:824] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:55.882676 9862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:55.878933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39703 (local address 127.18.71.126:40523)
0809 07:52:55.879348 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:55.879363 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:55.879397 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:52:55.880066 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:55.880085 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:55.880358 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:55.880733 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:55.881480 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:55.881486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:55.881504 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:55.881520 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:55.881572 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:55.881591 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:55.881700 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:55.881813 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 07:52:55.881973 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":134,"threads_started":1}
I20260809 07:52:55.884531 9859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147: No bootstrap required, opened a new log
I20260809 07:52:55.885356 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:55.888511 9859 raft_consensus.cc:348] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40523 } }
I20260809 07:52:55.888782 9859 raft_consensus.cc:374] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:55.888898 9859 raft_consensus.cc:729] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34ad40a6dbc4e16a06b6c3649c38147, State: Initialized, Role: FOLLOWER
I20260809 07:52:55.889007 9862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:55.887365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45259 (local address 127.18.71.126:40523)
0809 07:52:55.887474 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:55.887478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:55.887633 (+ 155us) server_negotiation.cc:407] Connection header received
0809 07:52:55.887748 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:55.887750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:55.887776 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:55.887838 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:55.888535 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:55.888539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:55.888541 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:55.888553 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:55.888581 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:55.888591 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:55.888658 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:55.888797 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 07:52:55.888837 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:52:55.889357 9859 consensus_queue.cc:260] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40523 } }
I20260809 07:52:55.889504 9859 raft_consensus.cc:388] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:55.889570 9859 raft_consensus.cc:482] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:55.889663 9859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:55.890522 9859 raft_consensus.cc:504] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40523 } }
I20260809 07:52:55.890846 9859 leader_election.cc:302] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d34ad40a6dbc4e16a06b6c3649c38147; no voters:
I20260809 07:52:55.891165 9859 leader_election.cc:288] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:55.891225 9870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:55.891444 9870 raft_consensus.cc:686] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [term 1 LEADER]: Becoming Leader. State: Replica: d34ad40a6dbc4e16a06b6c3649c38147, State: Running, Role: LEADER
I20260809 07:52:55.891820 9870 consensus_queue.cc:237] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40523 } }
I20260809 07:52:55.892361 9859 sys_catalog.cc:564] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:55.893714 9871 sys_catalog.cc:455] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40523 } } }
I20260809 07:52:55.893857 9871 sys_catalog.cc:458] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:55.894268 9883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:55.893743 9873 sys_catalog.cc:455] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d34ad40a6dbc4e16a06b6c3649c38147. Latest consensus state: current_term: 1 leader_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d34ad40a6dbc4e16a06b6c3649c38147" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 40523 } } }
I20260809 07:52:55.895637 9873 sys_catalog.cc:458] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147 [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:55.897521 9883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:55.902806 9883 catalog_manager.cc:1347] Generated new cluster ID: b66ab4edff7a4dc789a7a3e05ee615dc
I20260809 07:52:55.902868 9883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:55.920084 9883 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:55.920403 9883 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:55.930536 9883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d34ad40a6dbc4e16a06b6c3649c38147: Generated new TSK 0
I20260809 07:52:55.931186 9883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:55.941605 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9787
2026-08-09T07:52:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:56 dist-test-slave-10pc krb5kdc[9894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:56 dist-test-slave-10pc krb5kdc[9894](info): set up 2 sockets
Aug 09 07:52:56 dist-test-slave-10pc krb5kdc[9894](info): commencing operation
krb5kdc: starting...
W20260809 07:52:57.051218 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:57 dist-test-slave-10pc krb5kdc[9894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:57.183249 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46765
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:34011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:46765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:57.300606 9909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:57.300873 9909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:57.300957 9909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:57.305260 9909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:57.305327 9909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:57.305347 9909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:57.305364 9909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:57.305380 9909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:57.310818 9909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:34011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:46765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:57.311889 9909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:57.313037 9909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:57.321542 9914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:57.321578 9915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:57.321663 9909 server_base.cc:1034] running on GCE node
W20260809 07:52:57.321563 9917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:57.322384 9909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:57.323495 9909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:57.324687 9909 hybrid_clock.cc:648] HybridClock initialized: now 1786261977324657 us; error 42 us; skew 500 ppm
Aug 09 07:52:57 dist-test-slave-10pc krb5kdc[9894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261977, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:57.327638 9909 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:57.328675 9909 webserver.cc:460] Webserver started at http://127.18.71.126:38119/ using document root <none> and password file <none>
I20260809 07:52:57.329263 9909 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:57.329353 9909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:57.329591 9909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:57.331602 9909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "edd5c949da3b4d82bb464050a88cbdde"
format_stamp: "Formatted at 2026-08-09 07:52:57 on dist-test-slave-10pc"
server_key: "f1d9f22b4b608cd916c42711519960d6"
server_key_iv: "e39dcccdee6c926fec196594eea3911e"
server_key_version: "encryptionkey@0"
I20260809 07:52:57.332183 9909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "edd5c949da3b4d82bb464050a88cbdde"
format_stamp: "Formatted at 2026-08-09 07:52:57 on dist-test-slave-10pc"
server_key: "f1d9f22b4b608cd916c42711519960d6"
server_key_iv: "e39dcccdee6c926fec196594eea3911e"
server_key_version: "encryptionkey@0"
I20260809 07:52:57.335856 9909 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 07:52:57.338358 9924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:57.339634 9909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:52:57.339775 9909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "edd5c949da3b4d82bb464050a88cbdde"
format_stamp: "Formatted at 2026-08-09 07:52:57 on dist-test-slave-10pc"
server_key: "f1d9f22b4b608cd916c42711519960d6"
server_key_iv: "e39dcccdee6c926fec196594eea3911e"
server_key_version: "encryptionkey@0"
I20260809 07:52:57.339885 9909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:57.351080 9909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:57.354315 9909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:57.354522 9909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:57.361647 9909 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:46765
I20260809 07:52:57.361725 9981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:46765 every 8 connection(s)
I20260809 07:52:57.362607 9909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:57.365526 9983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:57.370247 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 9909
I20260809 07:52:57.370396 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:57.370708 18717 external_mini_cluster.cc:1428] Setting key dbf3d801614aa6f33cee0d3b7bb34afc
I20260809 07:52:57.371789 9983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde: Bootstrap starting.
I20260809 07:52:57.374204 9983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:57.375027 9983 log.cc:824] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:57.376971 9983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde: No bootstrap required, opened a new log
Aug 09 07:52:57 dist-test-slave-10pc krb5kdc[9894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:52:57.380829 9983 raft_consensus.cc:348] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd5c949da3b4d82bb464050a88cbdde" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46765 } }
I20260809 07:52:57.381091 9983 raft_consensus.cc:374] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:57.381165 9983 raft_consensus.cc:729] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edd5c949da3b4d82bb464050a88cbdde, State: Initialized, Role: FOLLOWER
I20260809 07:52:57.381609 9983 consensus_queue.cc:260] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd5c949da3b4d82bb464050a88cbdde" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46765 } }
I20260809 07:52:57.381770 9983 raft_consensus.cc:388] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:57.381860 9983 raft_consensus.cc:482] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:57.381989 9983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:57.382853 9983 raft_consensus.cc:504] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd5c949da3b4d82bb464050a88cbdde" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46765 } }
I20260809 07:52:57.383236 9983 leader_election.cc:302] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edd5c949da3b4d82bb464050a88cbdde; no voters:
I20260809 07:52:57.383548 9983 leader_election.cc:288] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:57.383680 9988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:57.383939 9988 raft_consensus.cc:686] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [term 1 LEADER]: Becoming Leader. State: Replica: edd5c949da3b4d82bb464050a88cbdde, State: Running, Role: LEADER
I20260809 07:52:57.384337 9988 consensus_queue.cc:237] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd5c949da3b4d82bb464050a88cbdde" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46765 } }
I20260809 07:52:57.384932 9983 sys_catalog.cc:564] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:57.386194 9989 sys_catalog.cc:455] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edd5c949da3b4d82bb464050a88cbdde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd5c949da3b4d82bb464050a88cbdde" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46765 } } }
I20260809 07:52:57.386327 9989 sys_catalog.cc:458] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:57.386577 9989 sys_catalog.cc:455] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [sys.catalog]: SysCatalogTable state changed. Reason: New leader edd5c949da3b4d82bb464050a88cbdde. Latest consensus state: current_term: 1 leader_uuid: "edd5c949da3b4d82bb464050a88cbdde" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edd5c949da3b4d82bb464050a88cbdde" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46765 } } }
I20260809 07:52:57.386688 9989 sys_catalog.cc:458] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:57.386621 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:57.372211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37515 (local address 127.18.71.126:46765)
0809 07:52:57.372685 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:57.372700 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:57.372735 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:52:57.373375 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:57.373393 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:57.373739 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:57.374078 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:57.374894 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:57.375884 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:57.376817 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:57.377331 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:57.377499 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:57.379952 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:57.379957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:57.379971 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:57.379997 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:57.382300 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:57.382768 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:57.382780 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:57.382786 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:57.382895 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:57.383373 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:57.383377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:57.383381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:57.384042 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:57.384979 (+ 937us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:57.385144 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 07:52:57.385469 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":175,"threads_started":1}
I20260809 07:52:57.390280 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 07:52:57.390358 10004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:57.390425 10004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:57.390516 9998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:57.391491 9998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:57.397985 9998 catalog_manager.cc:1347] Generated new cluster ID: 635b2eccccd340688f71cfec39e7b4f5
I20260809 07:52:57.398058 9998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:57.401484 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:57.392420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35679 (local address 127.18.71.126:46765)
0809 07:52:57.392567 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:57.392573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:57.392774 (+ 201us) server_negotiation.cc:407] Connection header received
0809 07:52:57.392933 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:57.392936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:57.392984 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:57.393067 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:57.393776 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:57.394673 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:57.395764 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:57.396220 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:57.396332 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:57.398223 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:57.398233 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:57.398236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:52:57.398257 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:57.399761 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:57.400202 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:57.400205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:57.400207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:57.400288 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:52:57.400620 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:52:57.400624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:52:57.400626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:52:57.400801 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:57.400938 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:57.401190 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 07:52:57.401307 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 07:52:57.411201 9998 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:57.412539 9998 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:57.421768 9998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edd5c949da3b4d82bb464050a88cbdde: Generated new TSK 0
I20260809 07:52:57.422607 9998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:57.428194 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 9909
2026-08-09T07:52:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:57 dist-test-slave-10pc krb5kdc[10015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:57 dist-test-slave-10pc krb5kdc[10015](info): set up 2 sockets
Aug 09 07:52:57 dist-test-slave-10pc krb5kdc[10015](info): commencing operation
krb5kdc: starting...
W20260809 07:52:58.512512 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:52:58 dist-test-slave-10pc krb5kdc[10015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261978, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:52:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:52:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:52:58.643051 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:38395
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:38395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:52:58.754418 10028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:52:58.754704 10028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:52:58.754792 10028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:52:58.758821 10028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:52:58.758899 10028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:52:58.758936 10028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:52:58.758991 10028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:52:58.759030 10028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:52:58.764209 10028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:38395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:38395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:52:58.765295 10028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:52:58.766347 10028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:52:58.774855 10035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:58.774896 10034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:52:58.774859 10037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:52:58.775089 10028 server_base.cc:1034] running on GCE node
I20260809 07:52:58.775777 10028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:52:58.776958 10028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:52:58.778152 10028 hybrid_clock.cc:648] HybridClock initialized: now 1786261978778137 us; error 26 us; skew 500 ppm
Aug 09 07:52:58 dist-test-slave-10pc krb5kdc[10015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261978, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:52:58.780959 10028 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:52:58.781865 10028 webserver.cc:460] Webserver started at http://127.18.71.126:39023/ using document root <none> and password file <none>
I20260809 07:52:58.782423 10028 fs_manager.cc:359] Metadata directory not provided
I20260809 07:52:58.782513 10028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:52:58.782744 10028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:52:58.784840 10028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "df10039f636947dd8621cf20a315497a"
format_stamp: "Formatted at 2026-08-09 07:52:58 on dist-test-slave-10pc"
server_key: "6b943fc288d3e6f300452d3cc8248d30"
server_key_iv: "68473b5bf17ffabbbf50d2a645cd19e4"
server_key_version: "encryptionkey@0"
I20260809 07:52:58.785414 10028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "df10039f636947dd8621cf20a315497a"
format_stamp: "Formatted at 2026-08-09 07:52:58 on dist-test-slave-10pc"
server_key: "6b943fc288d3e6f300452d3cc8248d30"
server_key_iv: "68473b5bf17ffabbbf50d2a645cd19e4"
server_key_version: "encryptionkey@0"
I20260809 07:52:58.789016 10028 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:52:58.791325 10044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:52:58.792583 10028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:52:58.792712 10028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "df10039f636947dd8621cf20a315497a"
format_stamp: "Formatted at 2026-08-09 07:52:58 on dist-test-slave-10pc"
server_key: "6b943fc288d3e6f300452d3cc8248d30"
server_key_iv: "68473b5bf17ffabbbf50d2a645cd19e4"
server_key_version: "encryptionkey@0"
I20260809 07:52:58.792819 10028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:52:58.814097 10028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:52:58.817360 10028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:52:58.817566 10028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:52:58.824713 10028 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:38395
I20260809 07:52:58.824738 10104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:38395 every 8 connection(s)
I20260809 07:52:58.825659 10028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:52:58.828486 10105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:52:58.829349 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10028
I20260809 07:52:58.829485 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:52:58.829782 18717 external_mini_cluster.cc:1428] Setting key 41be15e8a2f9ccd92a6f0716e20ea71a
I20260809 07:52:58.834532 10105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a: Bootstrap starting.
I20260809 07:52:58.836865 10105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a: Neither blocks nor log segments found. Creating new log.
I20260809 07:52:58.837602 10105 log.cc:824] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a: Log is configured to *not* fsync() on all Append() calls
I20260809 07:52:58.838793 10108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:58.831186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36255 (local address 127.18.71.126:38395)
0809 07:52:58.831625 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:58.831639 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:58.831674 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:52:58.832312 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:58.832330 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:58.832734 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:58.833085 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:58.833482 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:58.834421 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:58.835393 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:58.835862 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:58.836029 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:58.836917 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:58.836928 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:58.836940 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:58.837372 (+ 432us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:58.837428 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:58.837451 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:58.837563 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:58.837736 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 07:52:58.837943 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":124,"threads_started":1}
I20260809 07:52:58.839752 10105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a: No bootstrap required, opened a new log
I20260809 07:52:58.841650 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:52:58.843861 10105 raft_consensus.cc:348] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df10039f636947dd8621cf20a315497a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38395 } }
I20260809 07:52:58.844111 10105 raft_consensus.cc:374] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:52:58.844203 10105 raft_consensus.cc:729] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df10039f636947dd8621cf20a315497a, State: Initialized, Role: FOLLOWER
I20260809 07:52:58.844676 10105 consensus_queue.cc:260] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df10039f636947dd8621cf20a315497a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38395 } }
I20260809 07:52:58.844841 10105 raft_consensus.cc:388] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:52:58.844905 10105 raft_consensus.cc:482] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:52:58.845018 10105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:52:58.845928 10105 raft_consensus.cc:504] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df10039f636947dd8621cf20a315497a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38395 } }
I20260809 07:52:58.846293 10105 leader_election.cc:302] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df10039f636947dd8621cf20a315497a; no voters:
I20260809 07:52:58.846624 10105 leader_election.cc:288] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:52:58.846720 10115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:52:58.846980 10115 raft_consensus.cc:686] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [term 1 LEADER]: Becoming Leader. State: Replica: df10039f636947dd8621cf20a315497a, State: Running, Role: LEADER
I20260809 07:52:58.847282 10115 consensus_queue.cc:237] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df10039f636947dd8621cf20a315497a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38395 } }
I20260809 07:52:58.847872 10105 sys_catalog.cc:564] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:52:58.848085 10108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:52:58.843560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57687 (local address 127.18.71.126:38395)
0809 07:52:58.843695 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 07:52:58.843701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:52:58.843886 (+ 185us) server_negotiation.cc:407] Connection header received
0809 07:52:58.844031 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:52:58.844034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:52:58.844083 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:52:58.844160 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:52:58.844521 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:58.845279 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:58.846095 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:52:58.846442 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:52:58.846585 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:52:58.847420 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:52:58.847424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:52:58.847427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:52:58.847438 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:52:58.847465 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:52:58.847476 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:52:58.847539 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 07:52:58.847909 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 07:52:58.847958 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 07:52:58.849017 10116 sys_catalog.cc:455] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [sys.catalog]: SysCatalogTable state changed. Reason: New leader df10039f636947dd8621cf20a315497a. Latest consensus state: current_term: 1 leader_uuid: "df10039f636947dd8621cf20a315497a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df10039f636947dd8621cf20a315497a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38395 } } }
I20260809 07:52:58.849238 10116 sys_catalog.cc:458] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [sys.catalog]: This master's current role is: LEADER
I20260809 07:52:58.849298 10117 sys_catalog.cc:455] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df10039f636947dd8621cf20a315497a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df10039f636947dd8621cf20a315497a" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 38395 } } }
I20260809 07:52:58.849402 10117 sys_catalog.cc:458] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a [sys.catalog]: This master's current role is: LEADER
W20260809 07:52:58.852813 10131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:52:58.852880 10131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:52:58.852958 10125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:52:58.853750 10125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:52:58.859366 10125 catalog_manager.cc:1347] Generated new cluster ID: e687cb412f484fedb60799d0cd576635
I20260809 07:52:58.859431 10125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:52:58.876117 10125 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:52:58.877116 10125 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:52:58.884433 10125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df10039f636947dd8621cf20a315497a: Generated new TSK 0
I20260809 07:52:58.885314 10125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:52:58.898013 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10028
2026-08-09T07:52:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:52:58 dist-test-slave-10pc krb5kdc[10137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:52:58 dist-test-slave-10pc krb5kdc[10137](info): set up 2 sockets
Aug 09 07:52:58 dist-test-slave-10pc krb5kdc[10137](info): commencing operation
krb5kdc: starting...
W20260809 07:52:59.976693 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:53:00 dist-test-slave-10pc krb5kdc[10137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261980, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:53:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:53:00.108714 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42457
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:42457
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:00.222126 10150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:00.222398 10150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:00.222493 10150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:00.226576 10150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:00.226665 10150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:53:00.226703 10150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:00.226760 10150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:00.226800 10150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:00.232133 10150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:42457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:42457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:53:00.233187 10150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:00.234212 10150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:53:00.244515 10159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:00.244539 10157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:00.244880 10156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:00.245044 10150 server_base.cc:1034] running on GCE node
I20260809 07:53:00.245651 10150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:00.246702 10150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:00.247860 10150 hybrid_clock.cc:648] HybridClock initialized: now 1786261980247831 us; error 44 us; skew 500 ppm
Aug 09 07:53:00 dist-test-slave-10pc krb5kdc[10137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261980, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:00.250545 10150 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:53:00.251412 10150 webserver.cc:460] Webserver started at http://127.18.71.126:37889/ using document root <none> and password file <none>
I20260809 07:53:00.251962 10150 fs_manager.cc:359] Metadata directory not provided
I20260809 07:53:00.252053 10150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:00.252290 10150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:00.254382 10150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "795280d11a604262829cbe99859e6274"
format_stamp: "Formatted at 2026-08-09 07:53:00 on dist-test-slave-10pc"
server_key: "cbfbbce235f84818a4be75fe9f2654e9"
server_key_iv: "8e41204c964cb78ea4a9cdc5654a3664"
server_key_version: "encryptionkey@0"
I20260809 07:53:00.255004 10150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "795280d11a604262829cbe99859e6274"
format_stamp: "Formatted at 2026-08-09 07:53:00 on dist-test-slave-10pc"
server_key: "cbfbbce235f84818a4be75fe9f2654e9"
server_key_iv: "8e41204c964cb78ea4a9cdc5654a3664"
server_key_version: "encryptionkey@0"
I20260809 07:53:00.258639 10150 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:53:00.261045 10166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:00.262271 10150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:53:00.262426 10150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "795280d11a604262829cbe99859e6274"
format_stamp: "Formatted at 2026-08-09 07:53:00 on dist-test-slave-10pc"
server_key: "cbfbbce235f84818a4be75fe9f2654e9"
server_key_iv: "8e41204c964cb78ea4a9cdc5654a3664"
server_key_version: "encryptionkey@0"
I20260809 07:53:00.262538 10150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:00.267140 10150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:53:00.270128 10150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:00.270319 10150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:00.277470 10150 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:42457
I20260809 07:53:00.277491 10222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:42457 every 8 connection(s)
I20260809 07:53:00.278403 10150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:00.281386 10223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:00.285542 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10150
I20260809 07:53:00.285688 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:53:00.285956 18717 external_mini_cluster.cc:1428] Setting key e1d196c81fd262328e945fd4b50c7ec3
I20260809 07:53:00.287355 10223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274: Bootstrap starting.
I20260809 07:53:00.289721 10223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:00.290546 10223 log.cc:824] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:00.291164 10226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:00.287381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57997 (local address 127.18.71.126:42457)
0809 07:53:00.287809 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:00.287824 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:00.287858 (+ 34us) server_negotiation.cc:407] Connection header received
0809 07:53:00.288559 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:00.288578 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:00.288855 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:00.289196 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:00.289975 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:00.289979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:00.289991 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:53:00.290001 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:00.290052 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:53:00.290071 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:00.290192 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:00.290302 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 07:53:00.290459 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":137,"threads_started":1}
I20260809 07:53:00.292551 10223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274: No bootstrap required, opened a new log
I20260809 07:53:00.294029 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:53:00.296715 10223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795280d11a604262829cbe99859e6274" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42457 } }
I20260809 07:53:00.297022 10223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 07:53:00.297091 10223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 795280d11a604262829cbe99859e6274, State: Initialized, Role: FOLLOWER
I20260809 07:53:00.297591 10223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795280d11a604262829cbe99859e6274" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42457 } }
I20260809 07:53:00.297771 10223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 07:53:00.297852 10223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 07:53:00.297853 10226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:00.296042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52343 (local address 127.18.71.126:42457)
0809 07:53:00.296167 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:00.296176 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:00.296306 (+ 130us) server_negotiation.cc:407] Connection header received
0809 07:53:00.296443 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:00.296445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:00.296521 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:00.296581 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:00.297406 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:00.297409 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:00.297411 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 07:53:00.297420 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:00.297443 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 07:53:00.297452 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:00.297520 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:00.297635 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 07:53:00.297666 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 07:53:00.297981 10223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:00.299028 10223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795280d11a604262829cbe99859e6274" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42457 } }
I20260809 07:53:00.299365 10223 leader_election.cc:302] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 795280d11a604262829cbe99859e6274; no voters:
I20260809 07:53:00.299674 10223 leader_election.cc:288] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 07:53:00.299796 10234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:53:00.300066 10234 raft_consensus.cc:686] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [term 1 LEADER]: Becoming Leader. State: Replica: 795280d11a604262829cbe99859e6274, State: Running, Role: LEADER
I20260809 07:53:00.300361 10234 consensus_queue.cc:237] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795280d11a604262829cbe99859e6274" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42457 } }
I20260809 07:53:00.301017 10223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:00.301561 10234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 795280d11a604262829cbe99859e6274. Latest consensus state: current_term: 1 leader_uuid: "795280d11a604262829cbe99859e6274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795280d11a604262829cbe99859e6274" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42457 } } }
I20260809 07:53:00.301697 10234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:00.302067 10240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:53:00.302037 10236 sys_catalog.cc:455] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "795280d11a604262829cbe99859e6274" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "795280d11a604262829cbe99859e6274" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 42457 } } }
I20260809 07:53:00.302134 10236 sys_catalog.cc:458] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:00.305226 10240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:53:00.310892 10240 catalog_manager.cc:1347] Generated new cluster ID: b7ad353dc9a84bbe8774bb83d5c0d2a3
I20260809 07:53:00.310961 10240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:53:00.321631 10240 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:53:00.321929 10240 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:53:00.327701 10240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 795280d11a604262829cbe99859e6274: Generated new TSK 0
I20260809 07:53:00.328521 10240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 07:53:00.339986 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10150
2026-08-09T07:53:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1443 ms)
[----------] 18 tests from AuthTokenIssuingTest (26401 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:53:00 dist-test-slave-10pc krb5kdc[10256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:53:00 dist-test-slave-10pc krb5kdc[10256](info): set up 2 sockets
Aug 09 07:53:00 dist-test-slave-10pc krb5kdc[10256](info): commencing operation
krb5kdc: starting...
W20260809 07:53:01.421928 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:53:01 dist-test-slave-10pc krb5kdc[10256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:53:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:53:01.551779 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46737
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:46737,127.18.71.125:32865,127.18.71.124:46335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:01.665470 10270 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 07:53:01.665783 10270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:01.665845 10270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:01.665926 10270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:01.669992 10270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:01.670064 10270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:53:01.670084 10270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:01.670109 10270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:01.670151 10270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:01.675554 10270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:46737,127.18.71.125:32865,127.18.71.124:46335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:46737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:53:01.676705 10270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:01.677747 10270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:53:01.686712 10279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:01.686702 10275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:01.686801 10270 server_base.cc:1034] running on GCE node
W20260809 07:53:01.686702 10276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:01.687635 10270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:01.689029 10270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:01.690214 10270 hybrid_clock.cc:648] HybridClock initialized: now 1786261981690185 us; error 41 us; skew 500 ppm
Aug 09 07:53:01 dist-test-slave-10pc krb5kdc[10256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:01.698578 10270 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:53:01.699570 10270 webserver.cc:460] Webserver started at http://127.18.71.126:34387/ using document root <none> and password file <none>
I20260809 07:53:01.700147 10270 fs_manager.cc:359] Metadata directory not provided
I20260809 07:53:01.700217 10270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:01.700398 10270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:01.702592 10270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "7cd9e92cfe6643cc8c0c064b04224de2"
format_stamp: "Formatted at 2026-08-09 07:53:01 on dist-test-slave-10pc"
server_key: "8514229a5aa5ff7ee4bb3fb990c5112b"
server_key_iv: "585f2e35e6f5f0d12ffe327bbe6ff552"
server_key_version: "encryptionkey@0"
I20260809 07:53:01.703128 10270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "7cd9e92cfe6643cc8c0c064b04224de2"
format_stamp: "Formatted at 2026-08-09 07:53:01 on dist-test-slave-10pc"
server_key: "8514229a5aa5ff7ee4bb3fb990c5112b"
server_key_iv: "585f2e35e6f5f0d12ffe327bbe6ff552"
server_key_version: "encryptionkey@0"
I20260809 07:53:01.706812 10270 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 07:53:01.709190 10286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:01.710467 10270 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 07:53:01.710616 10270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "7cd9e92cfe6643cc8c0c064b04224de2"
format_stamp: "Formatted at 2026-08-09 07:53:01 on dist-test-slave-10pc"
server_key: "8514229a5aa5ff7ee4bb3fb990c5112b"
server_key_iv: "585f2e35e6f5f0d12ffe327bbe6ff552"
server_key_version: "encryptionkey@0"
I20260809 07:53:01.710722 10270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:01.724785 10270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:53:01.727573 10270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:01.727773 10270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:01.734853 10270 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:46737
I20260809 07:53:01.735165 10341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:46737 every 8 connection(s)
I20260809 07:53:01.735849 10270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:01.738649 10342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:01.738967 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10270
I20260809 07:53:01.739151 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:53:01.739419 18717 external_mini_cluster.cc:1428] Setting key af3e08b0708fd554ce911593baef3b01
I20260809 07:53:01.742599 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.71.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.125@KRBTEST.COM" created.
Aug 09 07:53:01 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:53:01.758191 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.744966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43920 (local address 127.18.71.126:46737)
0809 07:53:01.745386 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:01.745408 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:01.745678 (+ 270us) server_negotiation.cc:407] Connection header received
0809 07:53:01.746721 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:01.746725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:01.746898 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:01.747085 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:01.747911 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.748927 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.749929 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.750309 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.750513 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.753114 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:01.753123 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:01.753139 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:01.753169 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:01.754925 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.755560 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.755565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.755566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.755679 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.756058 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.756060 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.756061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.756441 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:01.756628 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:01.757106 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 07:53:01.757394 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":275,"thread_start_us":144,"threads_started":1}
I20260809 07:53:01.758198 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.744967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46737 (local address 127.0.0.1:43920)
0809 07:53:01.745536 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.745572 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:01.746406 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:01.747083 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:01.747096 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:01.747528 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:01.747765 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.747778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.749061 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.749064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.749776 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.749785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.750508 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.750512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.750626 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.751228 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:01.751257 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:01.752943 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:01.755051 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.755064 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.755077 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.755442 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.755778 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.755783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.755787 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.755949 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.756626 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:01.756636 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:01.756828 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 07:53:01.757052 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 07:53:01.757498 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":101,"spinlock_wait_cycles":14208,"thread_start_us":154,"threads_started":1}
I20260809 07:53:01.760802 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:01.761541 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.761295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51206)
0809 07:53:01.761420 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.761488 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 07:53:01.761734 10287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
I20260809 07:53:01.762554 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.762357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51212)
0809 07:53:01.762457 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.762500 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 07:53:01.762748 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:32865: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab.
Entry for principal kudu/127.18.71.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab.
I20260809 07:53:01.767418 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:32865
--webserver_interface=127.18.71.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab
--principal=kudu/127.18.71.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:46737,127.18.71.125:32865,127.18.71.124:46335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:01.812095 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } attempt: 1
I20260809 07:53:01.813076 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.812754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51224)
0809 07:53:01.812922 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.812998 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 07:53:01.813812 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.813588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51228)
0809 07:53:01.813704 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.813756 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 07:53:01.813937 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:32865: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
W20260809 07:53:01.880388 10351 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 07:53:01.880728 10351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:01.880801 10351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:01.880882 10351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 07:53:01.882102 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } attempt: 2
I20260809 07:53:01.882912 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.882614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51240)
0809 07:53:01.882768 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.882839 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 07:53:01.883687 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.883427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51256)
0809 07:53:01.883568 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.883621 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260809 07:53:01.883816 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:32865: Network error: Client connection negotiation failed: client connection to 127.18.71.125:32865: connect: Connection refused (error 111)
W20260809 07:53:01.885363 10351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:01.885435 10351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:53:01.885481 10351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:01.885536 10351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:01.885574 10351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:01.890676 10351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:46737,127.18.71.125:32865,127.18.71.124:46335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab
--principal=kudu/127.18.71.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:32865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.71.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:53:01.891713 10351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:01.892794 10351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:53:01.900981 10356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:01.901010 10357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:01.901000 10359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:01.901288 10351 server_base.cc:1034] running on GCE node
I20260809 07:53:01.901794 10351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:01.902848 10351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:01.904031 10351 hybrid_clock.cc:648] HybridClock initialized: now 1786261981903993 us; error 48 us; skew 500 ppm
Aug 09 07:53:01 dist-test-slave-10pc krb5kdc[10256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:01.906724 10351 init.cc:377] Logged in from keytab as kudu/127.18.71.125@KRBTEST.COM (short username kudu)
I20260809 07:53:01.907615 10351 webserver.cc:460] Webserver started at http://127.18.71.125:38063/ using document root <none> and password file <none>
I20260809 07:53:01.908174 10351 fs_manager.cc:359] Metadata directory not provided
I20260809 07:53:01.908263 10351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:01.908555 10351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:01.910667 10351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data/instance:
uuid: "c20f16ca2b79456eac1bbcae07809848"
format_stamp: "Formatted at 2026-08-09 07:53:01 on dist-test-slave-10pc"
server_key: "c6ec6da9b3979d9779656d153a0d6825"
server_key_iv: "544b4de4adf57fdaba4e24fa0e55ba83"
server_key_version: "encryptionkey@0"
I20260809 07:53:01.911319 10351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/wal/instance:
uuid: "c20f16ca2b79456eac1bbcae07809848"
format_stamp: "Formatted at 2026-08-09 07:53:01 on dist-test-slave-10pc"
server_key: "c6ec6da9b3979d9779656d153a0d6825"
server_key_iv: "544b4de4adf57fdaba4e24fa0e55ba83"
server_key_version: "encryptionkey@0"
I20260809 07:53:01.915066 10351 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:53:01.917769 10366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:01.919096 10351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 07:53:01.919222 10351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/wal
uuid: "c20f16ca2b79456eac1bbcae07809848"
format_stamp: "Formatted at 2026-08-09 07:53:01 on dist-test-slave-10pc"
server_key: "c6ec6da9b3979d9779656d153a0d6825"
server_key_iv: "544b4de4adf57fdaba4e24fa0e55ba83"
server_key_version: "encryptionkey@0"
I20260809 07:53:01.919325 10351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:01.939656 10351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:53:01.942471 10351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:01.942695 10351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:01.950023 10351 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.125:32865
I20260809 07:53:01.950322 10420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.125:32865 every 8 connection(s)
I20260809 07:53:01.951022 10351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
I20260809 07:53:01.953768 10421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:01.954856 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10351
I20260809 07:53:01.954991 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-1/wal/instance
I20260809 07:53:01.955333 18717 external_mini_cluster.cc:1428] Setting key ecc6478399bdb7bd534f473f1027420f
I20260809 07:53:01.958776 10421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.71.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.124@KRBTEST.COM" created.
Aug 09 07:53:01 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:53:01.973578 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.960768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43922 (local address 127.18.71.126:46737)
0809 07:53:01.960907 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:01.960911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:01.961887 (+ 976us) server_negotiation.cc:407] Connection header received
0809 07:53:01.962832 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:01.962836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:01.962909 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:01.963005 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:01.964142 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.964808 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.966092 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.966350 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.966471 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.969382 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:01.969388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:01.969393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:01.969426 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:01.971189 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.971718 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.971725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.971729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.971843 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.972258 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.972264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.972268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.972549 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:01.972662 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:01.973178 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 07:53:01.973333 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:53:01.974169 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.961249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46737 (local address 127.0.0.1:43922)
0809 07:53:01.961738 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.961780 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:01.962606 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:01.963185 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:01.963192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:01.963516 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:01.963987 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.964001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.964961 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.964964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.965955 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.965966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.966510 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.966514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.966692 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.967337 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:01.967354 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:01.969200 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:01.971323 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.971328 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.971341 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.971589 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.971944 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.971946 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.971948 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.972143 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.972661 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:01.972667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:01.972903 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 07:53:01.973061 (+ 158us) client_negotiation.cc:240] Negotiation successful
0809 07:53:01.973314 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":131,"threads_started":1}
I20260809 07:53:01.974973 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } attempt: 3
I20260809 07:53:01.975968 10421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:01 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
Aug 09 07:53:01 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab.
Entry for principal kudu/127.18.71.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab.
I20260809 07:53:01.985803 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:46335
--webserver_interface=127.18.71.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab
--principal=kudu/127.18.71.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:32941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:46737,127.18.71.125:32865,127.18.71.124:46335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:01.986603 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.975390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51258)
0809 07:53:01.975558 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.975573 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:01.975654 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:01.976306 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:01.976309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:01.976526 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:01.976641 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.976648 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.977916 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.977922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.978915 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.978924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.979733 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.979738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.979851 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.980545 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:01.980556 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:01.982247 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:01.984798 (+ 2551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.984803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.984805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.985024 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.985385 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.985388 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.985390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.985497 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.986058 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:01.986064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:01.986171 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 07:53:01.986258 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 07:53:01.986414 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 07:53:01.986688 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.976438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51274)
0809 07:53:01.976546 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.976560 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:01.976915 (+ 355us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:01.977169 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:01.977172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:01.977335 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:01.977434 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.977440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.978269 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.978272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.978998 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:01.979007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.979696 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.979700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:01.979831 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.980419 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:01.980438 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:01.981828 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:01.984780 (+ 2952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.984786 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.984791 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.985072 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.985628 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:01.985634 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:01.985639 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:01.985738 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:01.986210 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:01.986213 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:01.986298 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 07:53:01.986399 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 07:53:01.986531 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":6}
I20260809 07:53:01.987547 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.975495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51258 (local address 127.18.71.125:32865)
0809 07:53:01.975879 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:01.975892 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:01.975912 (+ 20us) server_negotiation.cc:407] Connection header received
0809 07:53:01.975951 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:01.975955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:01.976164 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:01.976304 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:01.976773 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.977803 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.979054 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.979594 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.979755 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.982411 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:01.982416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:01.982420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:01.982449 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:01.984625 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.985168 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.985177 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.985179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.985284 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.985597 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.985599 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.985600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.985927 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:01.986799 (+ 872us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:01.986879 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 07:53:01.987376 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":502,"server-negotiator.queue_time_us":293,"spinlock_wait_cycles":23936,"thread_start_us":167,"threads_started":1}
I20260809 07:53:01.987866 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.976660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51274 (local address 127.18.71.125:32865)
0809 07:53:01.976966 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:01.976973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:01.976984 (+ 11us) server_negotiation.cc:407] Connection header received
0809 07:53:01.977014 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:01.977016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:01.977049 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:01.977123 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:01.977538 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.978156 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.979114 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:01.979586 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:01.979718 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:01.981966 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:01.981979 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:01.981993 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:01.982016 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:01.984666 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.985192 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.985194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.985196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.985294 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:01.985839 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:01.985842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:01.985844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:01.986035 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:01.986170 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:01.986531 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 07:53:01.987304 (+ 773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":244,"thread_start_us":153,"threads_started":1}
I20260809 07:53:01.989718 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:01.990101 10421 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:01.990484 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.990264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41806)
0809 07:53:01.990368 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.990419 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 07:53:01.990859 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.990510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41812)
0809 07:53:01.990695 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.990777 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 07:53:01.991107 10370 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
I20260809 07:53:01.991277 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.991004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41826)
0809 07:53:01.991156 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.991205 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 07:53:01.991551 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46335: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
I20260809 07:53:01.992012 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:01.991821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41834)
0809 07:53:01.991924 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:01.991966 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 07:53:01.992218 10421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46335: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
I20260809 07:53:02.022713 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } attempt: 1
I20260809 07:53:02.023622 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.023292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41842)
0809 07:53:02.023468 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.023541 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 07:53:02.024392 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.024170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41852)
0809 07:53:02.024283 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.024336 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 07:53:02.024546 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46335: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
I20260809 07:53:02.041462 10421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } attempt: 1
I20260809 07:53:02.042274 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.041983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41858)
0809 07:53:02.042133 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.042199 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 07:53:02.042976 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.042764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41872)
0809 07:53:02.042873 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.042923 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 07:53:02.043115 10421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46335: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
I20260809 07:53:02.099714 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } attempt: 2
I20260809 07:53:02.100576 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.100221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41882)
0809 07:53:02.100373 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.100445 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 07:53:02.101284 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.101071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41888)
0809 07:53:02.101183 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.101235 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 07:53:02.101397 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46335: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
W20260809 07:53:02.104063 10429 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 07:53:02.104382 10429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:02.104444 10429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:02.104562 10429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:02.108748 10429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:02.108827 10429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:53:02.108880 10429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:02.108935 10429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:02.108973 10429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:02.111276 10421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } attempt: 2
I20260809 07:53:02.112082 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.111771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41898)
0809 07:53:02.111934 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.111998 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 07:53:02.112840 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.112624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41910)
0809 07:53:02.112728 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.112779 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 07:53:02.112972 10421 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46335: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46335: connect: Connection refused (error 111)
I20260809 07:53:02.114836 10429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:32941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:46737,127.18.71.125:32865,127.18.71.124:46335
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab
--principal=kudu/127.18.71.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:46335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.71.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:53:02.115880 10429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:02.116966 10429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:53:02.125692 10439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:02.125654 10436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:02.125654 10435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:02.125735 10429 server_base.cc:1034] running on GCE node
I20260809 07:53:02.126474 10429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:02.127570 10429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:02.128736 10429 hybrid_clock.cc:648] HybridClock initialized: now 1786261982128702 us; error 45 us; skew 500 ppm
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:02.131376 10429 init.cc:377] Logged in from keytab as kudu/127.18.71.124@KRBTEST.COM (short username kudu)
I20260809 07:53:02.132225 10429 webserver.cc:460] Webserver started at http://127.18.71.124:41241/ using document root <none> and password file <none>
I20260809 07:53:02.132851 10429 fs_manager.cc:359] Metadata directory not provided
I20260809 07:53:02.132941 10429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:02.133168 10429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:02.135218 10429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data/instance:
uuid: "e028d40779d4409ab1b29bff057f248e"
format_stamp: "Formatted at 2026-08-09 07:53:02 on dist-test-slave-10pc"
server_key: "4f37d1d4bc6117352ab0b8a747c1fc59"
server_key_iv: "9b7e38062c66ee2886c430cf0b6ef8d3"
server_key_version: "encryptionkey@0"
I20260809 07:53:02.135859 10429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/wal/instance:
uuid: "e028d40779d4409ab1b29bff057f248e"
format_stamp: "Formatted at 2026-08-09 07:53:02 on dist-test-slave-10pc"
server_key: "4f37d1d4bc6117352ab0b8a747c1fc59"
server_key_iv: "9b7e38062c66ee2886c430cf0b6ef8d3"
server_key_version: "encryptionkey@0"
I20260809 07:53:02.139523 10429 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:53:02.141927 10446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:02.143138 10429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:53:02.143285 10429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/wal
uuid: "e028d40779d4409ab1b29bff057f248e"
format_stamp: "Formatted at 2026-08-09 07:53:02 on dist-test-slave-10pc"
server_key: "4f37d1d4bc6117352ab0b8a747c1fc59"
server_key_iv: "9b7e38062c66ee2886c430cf0b6ef8d3"
server_key_version: "encryptionkey@0"
I20260809 07:53:02.143394 10429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:02.171118 10429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:53:02.174041 10429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:02.174253 10429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:02.181113 10429 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.124:46335
I20260809 07:53:02.181394 10500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.124:46335 every 8 connection(s)
I20260809 07:53:02.182087 10429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
I20260809 07:53:02.184422 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10429
I20260809 07:53:02.184577 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786261695308648-18717-0/minicluster-data/master-2/wal/instance
I20260809 07:53:02.184906 18717 external_mini_cluster.cc:1428] Setting key 651dfbfe964b3d1f009a928d6debd673
I20260809 07:53:02.185150 10501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:02.190348 10501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:53:02.200253 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.186242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.18.71.124:46335)
0809 07:53:02.186668 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.186689 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.186730 (+ 41us) server_negotiation.cc:407] Connection header received
0809 07:53:02.187544 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.187568 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.187983 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.188345 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.189081 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.190577 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.191590 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.192187 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.192415 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.195082 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.195087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.195095 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.195122 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.197020 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.197525 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.197530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.197535 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.197651 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.197976 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.197980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.197983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.198417 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.198656 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.198932 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.199218 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":111,"threads_started":1}
I20260809 07:53:02.200551 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } attempt: 3
I20260809 07:53:02.202314 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.192348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.18.71.126:46737)
0809 07:53:02.192528 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.192533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.192906 (+ 373us) server_negotiation.cc:407] Connection header received
0809 07:53:02.193183 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.193190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.193254 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.193348 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.194163 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.194755 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.195730 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.195972 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.196085 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.198358 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.198362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.198364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.198391 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.200028 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.200534 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.200541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.200543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.200865 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.201299 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.201303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.201305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.201543 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.201658 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.202034 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.202160 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 07:53:02.202346 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.192255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:46737 (local address 127.0.0.1:43930)
0809 07:53:02.192774 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.192825 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:02.193025 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:02.193405 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:02.193410 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:02.193775 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:02.194007 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.194017 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.194884 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.194888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.195585 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.195594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.196099 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.196103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.196233 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.196752 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:02.196763 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:02.198221 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:02.200141 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.200149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.200153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.200390 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.200965 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.200971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.200976 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.201202 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.201680 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:02.201686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:02.201798 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 07:53:02.201964 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 07:53:02.202160 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":108,"threads_started":1}
I20260809 07:53:02.204087 10501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:02.204164 10421 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } attempt: 3
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:53:02.210901 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.201001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41924)
0809 07:53:02.201157 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.201172 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:02.201244 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:02.201473 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:02.201476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:02.201676 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:02.201768 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.201775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.203109 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.203112 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.203803 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.203813 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.204379 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.204385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.204524 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.205032 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:02.205048 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:02.206576 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:02.208864 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.208867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.208869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.209474 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.209805 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.209810 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.209815 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.209903 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.210377 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:02.210383 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:02.210488 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 07:53:02.210581 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 07:53:02.210707 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 07:53:02.211452 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.201095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41924 (local address 127.18.71.124:46335)
0809 07:53:02.201191 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.201195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.201210 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:53:02.201327 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.201333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.201373 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.201454 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.201889 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.203011 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.203927 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.204242 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.206957 (+ 2715us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.207056 (+ 99us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.207059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.207062 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.207084 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.208754 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.209568 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.209572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.209573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.209691 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.210024 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.210027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.210029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.210254 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.211067 (+ 813us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.211160 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.211271 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 07:53:02.214448 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.202996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.18.71.126:46737)
0809 07:53:02.203236 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.203240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.203252 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:53:02.203284 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.203286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.203323 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.203413 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.204006 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.204719 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.205812 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.206131 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.206240 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.209014 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.209017 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.209020 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.209045 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.212062 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.212622 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.212625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.212626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.212714 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.213052 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.213058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.213061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.213274 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.213510 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.214151 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.214282 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":178}
I20260809 07:53:02.215569 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.204799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.18.71.125:32865)
0809 07:53:02.205138 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.205142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.205224 (+ 82us) server_negotiation.cc:407] Connection header received
0809 07:53:02.205392 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.205395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.205436 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.205484 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.206113 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.206904 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.208110 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.208518 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.208603 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.211403 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.211406 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.211409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.211430 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.213603 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.214108 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.214111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.214113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.214196 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.214532 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.214536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.214540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.214826 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.214899 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.215288 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.215418 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277}
I20260809 07:53:02.215935 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.204658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41938)
0809 07:53:02.204906 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.204920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:02.205011 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:02.205522 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:02.205525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:02.205718 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:02.205827 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.205834 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.207412 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.207416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.208310 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.208321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.209225 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.209229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.209322 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.209759 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:02.209780 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:02.211169 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:02.213950 (+ 2781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.213953 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.213955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.214182 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.214583 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.214597 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.214599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.214690 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.215473 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:02.215476 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:02.215569 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 07:53:02.215661 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 07:53:02.215792 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 07:53:02.215911 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2: Bootstrap starting.
I20260809 07:53:02.216006 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.204528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:32865 (local address 127.0.0.1:51278)
0809 07:53:02.205151 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.205165 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:02.205286 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:02.205654 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:02.205658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:02.205827 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:02.205938 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.205944 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.207044 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.207047 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.207952 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.207964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.208761 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.208764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.208873 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.209329 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:02.209345 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:02.211253 (+ 1908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:02.213729 (+ 2476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.213735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.213740 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.213990 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.214347 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.214350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.214352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.214427 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.214982 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:02.214983 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:02.215098 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 07:53:02.215704 (+ 606us) client_negotiation.cc:240] Negotiation successful
0809 07:53:02.215840 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548}
I20260809 07:53:02.216631 10501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:02.216919 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.204807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.18.71.124:46335)
0809 07:53:02.205303 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.205311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.205323 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:53:02.205354 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.205356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.205390 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.205924 (+ 534us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.206003 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.207263 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.208531 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.209086 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.209221 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.211301 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.211307 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.211313 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.211338 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.213829 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.214381 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.214385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.214387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.214475 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.215053 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.215056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.215057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.215365 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.216560 (+ 1195us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.216648 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.216755 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":196,"threads_started":1}
I20260809 07:53:02.218827 10342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:02.219733 10421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: Bootstrap starting.
I20260809 07:53:02.219727 10342 log.cc:824] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
I20260809 07:53:02.222121 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2: No bootstrap required, opened a new log
I20260809 07:53:02.222361 10421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:02.223282 10421 log.cc:824] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: Log is configured to *not* fsync() on all Append() calls
Aug 09 07:53:02 dist-test-slave-10pc krb5kdc[10256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261982, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:53:02.226065 10421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: No bootstrap required, opened a new log
I20260809 07:53:02.226979 10342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
I20260809 07:53:02.227232 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.215959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51288 (local address 127.18.71.125:32865)
0809 07:53:02.216587 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.216593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.216795 (+ 202us) server_negotiation.cc:407] Connection header received
0809 07:53:02.217167 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.217172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.217222 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.217277 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.218008 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.218475 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.219274 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.219500 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.219600 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.222614 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.222618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.222621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.222645 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.224664 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.225258 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.225261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.225263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.225347 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.225919 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.225923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.225925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.226418 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.226573 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.226950 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.227086 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557}
I20260809 07:53:02.227217 10342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd9e92cfe6643cc8c0c064b04224de2, State: Initialized, Role: FOLLOWER
I20260809 07:53:02.227802 10342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
I20260809 07:53:02.228852 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.217069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46335 (local address 127.0.0.1:41948)
0809 07:53:02.217367 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:02.217381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:02.217441 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:02.220539 (+ 3098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:02.220542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:02.220693 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:02.220767 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.220773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.221712 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.221715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.222518 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:02.222531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.223150 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.223154 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:02.223225 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.223766 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:02.223780 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:02.225159 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:02.226997 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.226999 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.227001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.227492 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.227763 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:02.227767 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:02.227770 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:02.227837 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:02.228313 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:02.228316 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:02.228400 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 07:53:02.228519 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 07:53:02.228647 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260809 07:53:02.229401 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.217251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.18.71.124:46335)
0809 07:53:02.220249 (+ 2998us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.220253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.220267 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:53:02.220317 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.220320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.220369 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.220441 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.220880 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.221595 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.222651 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.222941 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.223029 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.225289 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.225292 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.225295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.225319 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.226885 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.227605 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.227608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.227610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.227680 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.227914 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.227917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.227919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.228092 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.228178 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.229157 (+ 979us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.229265 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2919}
I20260809 07:53:02.229722 10342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:02.230486 10421 raft_consensus.cc:348] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
I20260809 07:53:02.230720 10421 raft_consensus.cc:729] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20f16ca2b79456eac1bbcae07809848, State: Initialized, Role: FOLLOWER
I20260809 07:53:02.231045 10421 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
I20260809 07:53:02.231839 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.232049 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:02.232446 10421 sys_catalog.cc:564] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:02.232620 10510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.232750 10510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:02.233464 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:53:02.233580 10501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: Bootstrap starting.
I20260809 07:53:02.234884 10315 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7cd9e92cfe6643cc8c0c064b04224de2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43940
I20260809 07:53:02.235172 10315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:53:02.235292 10315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [term 0 FOLLOWER]: Advancing to term 1
W20260809 07:53:02.236229 10523 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 07:53:02.236295 10501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: Neither blocks nor log segments found. Creating new log.
W20260809 07:53:02.236390 10523 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:02.236775 10315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
I20260809 07:53:02.237216 10501 log.cc:824] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:02.237748 10315 leader_election.cc:288] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [CANDIDATE]: Term 1 election: Requested vote from peers c20f16ca2b79456eac1bbcae07809848 (127.18.71.125:32865), e028d40779d4409ab1b29bff057f248e (127.18.71.124:46335)
I20260809 07:53:02.234966 10270 master_runner.cc:186] Error getting master registration for 127.18.71.125:32865: OK, instance_id { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" instance_seqno: 1786261981947927 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:02.238610 10393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c20f16ca2b79456eac1bbcae07809848"
I20260809 07:53:02.238703 10270 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:02.238830 10393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:02.240154 10393 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd9e92cfe6643cc8c0c064b04224de2 in term 1.
I20260809 07:53:02.240581 10287 leader_election.cc:302] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd9e92cfe6643cc8c0c064b04224de2, c20f16ca2b79456eac1bbcae07809848; no voters:
I20260809 07:53:02.240880 10510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:53:02.241681 10501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: No bootstrap required, opened a new log
I20260809 07:53:02.242106 10510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [term 1 LEADER]: Becoming Leader. State: Replica: 7cd9e92cfe6643cc8c0c064b04224de2, State: Running, Role: LEADER
I20260809 07:53:02.242638 10510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
I20260809 07:53:02.246446 10501 raft_consensus.cc:348] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
I20260809 07:53:02.246701 10501 raft_consensus.cc:729] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e028d40779d4409ab1b29bff057f248e, State: Initialized, Role: FOLLOWER
I20260809 07:53:02.247022 10501 consensus_queue.cc:260] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } }
W20260809 07:53:02.247282 10540 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:02.247594 10540 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:02.247889 10510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cd9e92cfe6643cc8c0c064b04224de2. Latest consensus state: current_term: 1 leader_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.248014 10510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:02.248418 10544 sys_catalog.cc:455] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.248600 10544 sys_catalog.cc:458] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:02.248903 10545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:53:02.249950 10545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:53:02.248533 10501 sys_catalog.cc:564] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:02.249596 10351 master_runner.cc:186] Error getting master registration for 127.18.71.124:46335: OK, instance_id { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" instance_seqno: 1786261982179233 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:02.252885 10351 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:02.253089 10474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e028d40779d4409ab1b29bff057f248e"
I20260809 07:53:02.253331 10474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:02.254896 10474 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd9e92cfe6643cc8c0c064b04224de2 in term 1.
I20260809 07:53:02.265401 10505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.248932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41962 (local address 127.18.71.124:46335)
0809 07:53:02.249046 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.249051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.252769 (+ 3718us) server_negotiation.cc:407] Connection header received
0809 07:53:02.253615 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.253621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.253677 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.253732 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.254765 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.255510 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.256347 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.256678 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.256781 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.258757 (+ 1976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.258761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.258764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.258790 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.260621 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.263948 (+ 3327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.263954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.263957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.264043 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.264358 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.264361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.264363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.264623 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.264733 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.265119 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.265233 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 07:53:02.266829 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.250417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51302 (local address 127.18.71.125:32865)
0809 07:53:02.250539 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.250543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.253435 (+ 2892us) server_negotiation.cc:407] Connection header received
0809 07:53:02.253482 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.253485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.253548 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.253624 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.257731 (+ 4107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.258483 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.260719 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.261053 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.261152 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.262854 (+ 1702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.262858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.262862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.262898 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.264730 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.265339 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.265344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.265346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.265441 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.265793 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.265798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.265801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.266042 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.266180 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.266557 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.266678 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 07:53:02.269315 10561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:02.269449 10561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:02.269536 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:02.249699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43942 (local address 127.18.71.126:46737)
0809 07:53:02.249803 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.249807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.253012 (+ 3205us) server_negotiation.cc:407] Connection header received
0809 07:53:02.253167 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.253170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.253233 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.253333 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.258960 (+ 5627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.259636 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.261716 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:02.262075 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:02.262170 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:02.264550 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:02.264554 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:02.264557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:02.264586 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:02.266343 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.268096 (+ 1753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.268103 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.268106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.268202 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:02.268529 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:02.268533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:02.268534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:02.268770 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:02.268896 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:02.269264 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 07:53:02.269381 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 07:53:02.269536 10474 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [term 1 FOLLOWER]: Refusing update from remote peer 7cd9e92cfe6643cc8c0c064b04224de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:02.270179 10510 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:53:02.270757 10393 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [term 1 FOLLOWER]: Refusing update from remote peer 7cd9e92cfe6643cc8c0c064b04224de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:02.274495 10544 sys_catalog.cc:455] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cd9e92cfe6643cc8c0c064b04224de2. Latest consensus state: current_term: 1 leader_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.274608 10544 sys_catalog.cc:458] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:02.274808 10510 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:53:02.278035 10541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.278167 10541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:02.278452 10549 mvcc.cc:205] Tried to move back new op lower bound from 7316529079339900928 to 7316529079269933056. Current Snapshot: MvccSnapshot[applied={T|T < 7316529079339900928}]
I20260809 07:53:02.278518 10544 sys_catalog.cc:455] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.278637 10544 sys_catalog.cc:458] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:02.279243 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cd9e92cfe6643cc8c0c064b04224de2. Latest consensus state: current_term: 1 leader_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.279366 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:02.280027 10563 mvcc.cc:205] Tried to move back new op lower bound from 7316529079339900928 to 7316529079269933056. Current Snapshot: MvccSnapshot[applied={T|T < 7316529079339900928}]
I20260809 07:53:02.286011 10545 catalog_manager.cc:1347] Generated new cluster ID: 5e57b526de8e4d57a9ea6aab92b90afd
I20260809 07:53:02.286087 10545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:53:02.302218 10510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.302362 10510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:02.303768 10516 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd9e92cfe6643cc8c0c064b04224de2" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 46737 } } peers { permanent_uuid: "c20f16ca2b79456eac1bbcae07809848" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 32865 } } peers { permanent_uuid: "e028d40779d4409ab1b29bff057f248e" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46335 } } }
I20260809 07:53:02.303889 10516 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:02.304008 10565 mvcc.cc:205] Tried to move back new op lower bound from 7316529079339900928 to 7316529079269933056. Current Snapshot: MvccSnapshot[applied={T|T < 7316529079339900928}]
I20260809 07:53:02.312363 10545 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:53:02.313674 10545 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:53:02.324205 10545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cd9e92cfe6643cc8c0c064b04224de2: Generated new TSK 0
I20260809 07:53:02.325120 10545 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 07:53:02.358963 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.355604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51310 (local address 127.18.71.125:32865)
0809 07:53:02.355740 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.355745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.357811 (+ 2066us) server_negotiation.cc:407] Connection header received
0809 07:53:02.358022 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.358026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.358080 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.358135 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.358820 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51310: BlockingRecv error: recv got EOF from 127.0.0.1:51310 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 07:53:02.360423 10345 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.356107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.18.71.126:46737)
0809 07:53:02.356218 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.356223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.359491 (+ 3268us) server_negotiation.cc:407] Connection header received
0809 07:53:02.359694 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.359697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.359749 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.359802 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.360300 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43956: BlockingRecv error: recv got EOF from 127.0.0.1:43956 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 07:53:02.361934 10505 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.356583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.18.71.124:46335)
0809 07:53:02.356697 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.356701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.361029 (+ 4328us) server_negotiation.cc:407] Connection header received
0809 07:53:02.361205 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.361208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.361258 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.361310 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.361809 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41972: BlockingRecv error: recv got EOF from 127.0.0.1:41972 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 07:53:02.366796 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.365155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51316 (local address 127.18.71.125:32865)
0809 07:53:02.365263 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.365267 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.365808 (+ 541us) server_negotiation.cc:407] Connection header received
0809 07:53:02.366037 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.366040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.366122 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.366179 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.366696 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51316: BlockingRecv error: recv got EOF from 127.0.0.1:51316 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 07:53:02.371822 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.370294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.18.71.125:32865)
0809 07:53:02.370387 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.370391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.370949 (+ 558us) server_negotiation.cc:407] Connection header received
0809 07:53:02.371144 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.371148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.371196 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.371244 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.371730 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51320: BlockingRecv error: recv got EOF from 127.0.0.1:51320 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 07:53:02.377348 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.375925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.18.71.125:32865)
0809 07:53:02.376017 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.376021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.376507 (+ 486us) server_negotiation.cc:407] Connection header received
0809 07:53:02.376703 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.376707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.376751 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.376799 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.377262 (+ 463us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51324: BlockingRecv error: recv got EOF from 127.0.0.1:51324 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 07:53:02.385032 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.383397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.18.71.125:32865)
0809 07:53:02.383536 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.383540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.384102 (+ 562us) server_negotiation.cc:407] Connection header received
0809 07:53:02.384282 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.384285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.384330 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.384380 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.384923 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51330: BlockingRecv error: recv got EOF from 127.0.0.1:51330 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 07:53:02.396723 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.395177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.18.71.125:32865)
0809 07:53:02.395274 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.395279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.395775 (+ 496us) server_negotiation.cc:407] Connection header received
0809 07:53:02.395961 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.395965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.396010 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.396063 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.396628 (+ 565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51332: BlockingRecv error: recv got EOF from 127.0.0.1:51332 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 07:53:02.417544 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.415411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.18.71.125:32865)
0809 07:53:02.415532 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.415537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.416096 (+ 559us) server_negotiation.cc:407] Connection header received
0809 07:53:02.416313 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.416317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.416376 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.416433 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.417428 (+ 995us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51334: BlockingRecv error: recv got EOF from 127.0.0.1:51334 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 07:53:02.454299 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.452501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51340 (local address 127.18.71.125:32865)
0809 07:53:02.452638 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.452643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.453249 (+ 606us) server_negotiation.cc:407] Connection header received
0809 07:53:02.453468 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.453472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.453530 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.453584 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.454182 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51340: BlockingRecv error: recv got EOF from 127.0.0.1:51340 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 07:53:02.522013 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.520823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.18.71.125:32865)
0809 07:53:02.520975 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.520980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.521178 (+ 198us) server_negotiation.cc:407] Connection header received
0809 07:53:02.521291 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.521293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.521340 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.521414 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.521891 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51350: BlockingRecv error: recv got EOF from 127.0.0.1:51350 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 07:53:02.653461 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.652139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51356 (local address 127.18.71.125:32865)
0809 07:53:02.652279 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.652283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.652521 (+ 238us) server_negotiation.cc:407] Connection header received
0809 07:53:02.652710 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.652713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.652765 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.652838 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.653318 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51356: BlockingRecv error: recv got EOF from 127.0.0.1:51356 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 07:53:02.912853 10427 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:02.911608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.18.71.125:32865)
0809 07:53:02.911757 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:02.911762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:02.911853 (+ 91us) server_negotiation.cc:407] Connection header received
0809 07:53:02.912034 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:02.912037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:02.912093 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:02.912185 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:02.912731 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51362: BlockingRecv error: recv got EOF from 127.0.0.1:51362 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 07:53:03.248919 10540 catalog_manager.cc:1259] Loaded cluster ID: 5e57b526de8e4d57a9ea6aab92b90afd
I20260809 07:53:03.249006 10540 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: loading cluster ID for follower catalog manager: success
I20260809 07:53:03.250718 10540 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: acquiring CA information for follower catalog manager: success
I20260809 07:53:03.251899 10540 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c20f16ca2b79456eac1bbcae07809848: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:53:03.270821 10561 catalog_manager.cc:1259] Loaded cluster ID: 5e57b526de8e4d57a9ea6aab92b90afd
I20260809 07:53:03.270903 10561 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: loading cluster ID for follower catalog manager: success
I20260809 07:53:03.272643 10561 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: acquiring CA information for follower catalog manager: success
I20260809 07:53:03.273813 10561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e028d40779d4409ab1b29bff057f248e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:53:03.431267 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:03.426953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.18.71.125:32865)
0809 07:53:03.427213 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:03.427218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:03.427311 (+ 93us) server_negotiation.cc:407] Connection header received
0809 07:53:03.427499 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:03.427502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:03.427648 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:03.427779 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:03.428049 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.428624 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.429497 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.429735 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.429881 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:03.430229 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:03.430703 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:03.430846 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:03.430974 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 07:53:03.431062 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":81,"threads_started":1}
I20260809 07:53:03.438030 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:03.434479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41984 (local address 127.18.71.124:46335)
0809 07:53:03.434708 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:03.434712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:03.434726 (+ 14us) server_negotiation.cc:407] Connection header received
0809 07:53:03.434836 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:03.434839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:03.434952 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:03.435051 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:03.435254 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.435752 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.436608 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.436821 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.436899 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:03.437152 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:03.437567 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:03.437661 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:03.437757 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 07:53:03.437824 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
I20260809 07:53:03.445077 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:03.441285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.18.71.124:46335)
0809 07:53:03.441392 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:03.441396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:03.441771 (+ 375us) server_negotiation.cc:407] Connection header received
0809 07:53:03.441924 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:03.441927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:03.441971 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:03.442040 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:03.442323 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.442920 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.443854 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.444195 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.444291 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:03.444638 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:03.444700 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:03.444779 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:03.444870 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 07:53:03.444917 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 07:53:03.445307 10643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:03.441067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51388 (local address 127.18.71.125:32865)
0809 07:53:03.441199 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:03.441204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:03.441542 (+ 338us) server_negotiation.cc:407] Connection header received
0809 07:53:03.441737 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:03.441740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:03.441795 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:03.441861 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:03.442113 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.442748 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.443948 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.444278 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.444346 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:03.444761 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:03.444841 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:03.444888 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:03.445093 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 07:53:03.445148 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 07:53:03.446178 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:03.441182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.18.71.126:46737)
0809 07:53:03.441413 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:03.441418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:03.441712 (+ 294us) server_negotiation.cc:407] Connection header received
0809 07:53:03.441976 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:03.441979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:03.442160 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:03.442235 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:03.442639 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.443443 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.444428 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:03.444808 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:03.444917 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:03.445361 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:03.445713 (+ 352us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:03.445776 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:03.445953 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 07:53:03.446026 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
I20260809 07:53:03.449075 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10270
I20260809 07:53:03.459312 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10351
I20260809 07:53:03.466796 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10429
2026-08-09T07:53:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3125 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 07:53:03 dist-test-slave-10pc krb5kdc[10660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 07:53:03 dist-test-slave-10pc krb5kdc[10660](info): set up 2 sockets
Aug 09 07:53:03 dist-test-slave-10pc krb5kdc[10660](info): commencing operation
krb5kdc: starting...
W20260809 07:53:04.549502 18717 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/test-user.keytab.
Aug 09 07:53:04 dist-test-slave-10pc krb5kdc[10660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T07:53:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T07:53:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.18.71.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
Entry for principal kudu/127.18.71.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab.
I20260809 07:53:04.677615 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:37381
--webserver_interface=127.18.71.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:37381,127.18.71.125:37991,127.18.71.124:46275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 07:53:04.789211 10673 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 07:53:04.789491 10673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:04.789548 10673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:04.789629 10673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 07:53:04.793715 10673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:04.793768 10673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:53:04.793788 10673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:04.793805 10673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:04.793857 10673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:04.799033 10673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:37381,127.18.71.125:37991,127.18.71.124:46275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.126.keytab
--principal=kudu/127.18.71.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.126:37381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.18.71.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:53:04.800038 10673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:04.801069 10673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:53:04.809623 10679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:04.809863 10673 server_base.cc:1034] running on GCE node
W20260809 07:53:04.809630 10678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:04.809619 10681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:04.810648 10673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:04.811683 10673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:04.812844 10673 hybrid_clock.cc:648] HybridClock initialized: now 1786261984812826 us; error 31 us; skew 500 ppm
Aug 09 07:53:04 dist-test-slave-10pc krb5kdc[10660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:04.815490 10673 init.cc:377] Logged in from keytab as kudu/127.18.71.126@KRBTEST.COM (short username kudu)
I20260809 07:53:04.816372 10673 webserver.cc:460] Webserver started at http://127.18.71.126:46549/ using document root <none> and password file <none>
I20260809 07:53:04.816941 10673 fs_manager.cc:359] Metadata directory not provided
I20260809 07:53:04.817035 10673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:04.817278 10673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:04.819268 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data/instance:
uuid: "06634d9dfb4c406a8ff032d454152ca1"
format_stamp: "Formatted at 2026-08-09 07:53:04 on dist-test-slave-10pc"
server_key: "a126d5adebbbef20c207f427b97fade5"
server_key_iv: "76ee48c9cf57c4ff74a96e7816713e3e"
server_key_version: "encryptionkey@0"
I20260809 07:53:04.819844 10673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/wal/instance:
uuid: "06634d9dfb4c406a8ff032d454152ca1"
format_stamp: "Formatted at 2026-08-09 07:53:04 on dist-test-slave-10pc"
server_key: "a126d5adebbbef20c207f427b97fade5"
server_key_iv: "76ee48c9cf57c4ff74a96e7816713e3e"
server_key_version: "encryptionkey@0"
I20260809 07:53:04.823570 10673 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:53:04.825917 10688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:04.827033 10673 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 07:53:04.827188 10673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/wal
uuid: "06634d9dfb4c406a8ff032d454152ca1"
format_stamp: "Formatted at 2026-08-09 07:53:04 on dist-test-slave-10pc"
server_key: "a126d5adebbbef20c207f427b97fade5"
server_key_iv: "76ee48c9cf57c4ff74a96e7816713e3e"
server_key_version: "encryptionkey@0"
I20260809 07:53:04.827296 10673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:04.851886 10673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:53:04.854945 10673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:04.855172 10673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:04.862457 10673 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.126:37381
I20260809 07:53:04.862763 10745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.126:37381 every 8 connection(s)
I20260809 07:53:04.863449 10673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/data/info.pb
I20260809 07:53:04.864225 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10673
I20260809 07:53:04.864355 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-0/wal/instance
I20260809 07:53:04.864698 18717 external_mini_cluster.cc:1428] Setting key 8b0cff87c191c50ae82dde0d935587cf
I20260809 07:53:04.867028 10746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:04.870945 10746 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.71.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.125@KRBTEST.COM" created.
Aug 09 07:53:04 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:53:04.887234 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:04.873310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37381 (local address 127.0.0.1:54816)
0809 07:53:04.873849 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:04.873899 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:04.874777 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:04.875482 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:04.875487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:04.875928 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:04.876169 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:04.876182 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:04.877711 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:04.877714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:04.878581 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:04.878593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:04.879242 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:04.879247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:04.879393 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:04.879984 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:04.880009 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:04.881896 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:04.884330 (+ 2434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:04.884339 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:04.884347 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:04.884644 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:04.885033 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:04.885035 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:04.885037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:04.885152 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:04.885912 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:04.885917 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:04.886036 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 07:53:04.886186 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 07:53:04.886429 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":142,"threads_started":1}
I20260809 07:53:04.887234 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:04.873309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.18.71.126:37381)
0809 07:53:04.873856 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:04.873874 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:04.874012 (+ 138us) server_negotiation.cc:407] Connection header received
0809 07:53:04.875088 (+ 1076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:04.875093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:04.875292 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:04.875483 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:04.876319 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:04.877552 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:04.878713 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:04.879073 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:04.879199 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:04.882087 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:04.882097 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:04.882126 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:04.882155 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:04.884216 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:04.884760 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:04.884767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:04.884770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:04.884915 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:04.885249 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:04.885252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:04.885256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:04.885794 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:04.885944 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:04.886309 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 07:53:04.886551 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":455,"spinlock_wait_cycles":48512,"thread_start_us":171,"threads_started":1}
I20260809 07:53:04.889645 10746 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:04.890347 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:04.890080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51484)
0809 07:53:04.890196 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:04.890261 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 07:53:04.890570 10689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
I20260809 07:53:04.891315 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:04.891146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51486)
0809 07:53:04.891230 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:04.891271 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 07:53:04.891554 10746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:37991: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab.
Entry for principal kudu/127.18.71.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab.
I20260809 07:53:04.895818 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:37991
--webserver_interface=127.18.71.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab
--principal=kudu/127.18.71.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:37381,127.18.71.125:37991,127.18.71.124:46275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:04.940802 10746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } attempt: 1
I20260809 07:53:04.941756 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:04.941456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51498)
0809 07:53:04.941602 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:04.941680 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 07:53:04.942526 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:04.942301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51506)
0809 07:53:04.942414 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:04.942468 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 07:53:04.942662 10746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:37991: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
W20260809 07:53:05.008018 10753 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 07:53:05.008363 10753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:05.008448 10753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:05.008545 10753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 07:53:05.010968 10746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } attempt: 2
I20260809 07:53:05.011878 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.011586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51512)
0809 07:53:05.011735 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.011804 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 07:53:05.012605 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.012360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51526)
0809 07:53:05.012502 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.012551 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 07:53:05.012727 10746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.125:37991: Network error: Client connection negotiation failed: client connection to 127.18.71.125:37991: connect: Connection refused (error 111)
W20260809 07:53:05.012843 10753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:05.012923 10753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:53:05.012969 10753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:05.013006 10753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:05.013042 10753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:05.018263 10753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:37381,127.18.71.125:37991,127.18.71.124:46275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.125.keytab
--principal=kudu/127.18.71.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.125:37991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.18.71.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:53:05.019392 10753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:05.020478 10753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:53:05.028761 10760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:05.028766 10762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:05.028864 10759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:05.029194 10753 server_base.cc:1034] running on GCE node
I20260809 07:53:05.029692 10753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:05.030874 10753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:05.032075 10753 hybrid_clock.cc:648] HybridClock initialized: now 1786261985032066 us; error 37 us; skew 500 ppm
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:05.034817 10753 init.cc:377] Logged in from keytab as kudu/127.18.71.125@KRBTEST.COM (short username kudu)
I20260809 07:53:05.035657 10753 webserver.cc:460] Webserver started at http://127.18.71.125:44209/ using document root <none> and password file <none>
I20260809 07:53:05.036185 10753 fs_manager.cc:359] Metadata directory not provided
I20260809 07:53:05.036243 10753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:05.036410 10753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:05.038489 10753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data/instance:
uuid: "3d325f792da240328024ec9d31384508"
format_stamp: "Formatted at 2026-08-09 07:53:05 on dist-test-slave-10pc"
server_key: "dec2784cb3c8a350f686cec717833ee4"
server_key_iv: "45411ead440fa393e6cdadc37be7ed32"
server_key_version: "encryptionkey@0"
I20260809 07:53:05.039016 10753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/wal/instance:
uuid: "3d325f792da240328024ec9d31384508"
format_stamp: "Formatted at 2026-08-09 07:53:05 on dist-test-slave-10pc"
server_key: "dec2784cb3c8a350f686cec717833ee4"
server_key_iv: "45411ead440fa393e6cdadc37be7ed32"
server_key_version: "encryptionkey@0"
I20260809 07:53:05.042843 10753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 07:53:05.045230 10769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:05.046448 10753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 07:53:05.046550 10753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/wal
uuid: "3d325f792da240328024ec9d31384508"
format_stamp: "Formatted at 2026-08-09 07:53:05 on dist-test-slave-10pc"
server_key: "dec2784cb3c8a350f686cec717833ee4"
server_key_iv: "45411ead440fa393e6cdadc37be7ed32"
server_key_version: "encryptionkey@0"
I20260809 07:53:05.046636 10753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:05.058182 10753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:53:05.060947 10753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:05.061101 10753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:05.068104 10753 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.125:37991
I20260809 07:53:05.068413 10826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.125:37991 every 8 connection(s)
I20260809 07:53:05.069177 10753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/data/info.pb
I20260809 07:53:05.071977 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:05.072510 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10753
I20260809 07:53:05.072674 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-1/wal/instance
I20260809 07:53:05.072978 18717 external_mini_cluster.cc:1428] Setting key f4e8526699e2897adcace4ed3da914ce
I20260809 07:53:05.076556 10827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.18.71.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.18.71.124@KRBTEST.COM" created.
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
I20260809 07:53:05.092247 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.079053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.18.71.126:37381)
0809 07:53:05.079197 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.079206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.080300 (+ 1094us) server_negotiation.cc:407] Connection header received
0809 07:53:05.081373 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.081376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.081444 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.081523 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.082693 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.083262 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.084434 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.084727 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.084842 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.087396 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.087399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.087401 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.087428 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.089922 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.090434 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.090439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.090443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.090551 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.090911 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.090916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.090919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.091190 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.091305 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.091833 (+ 528us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.091999 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 07:53:05.092983 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.079581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37381 (local address 127.0.0.1:54822)
0809 07:53:05.080145 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.080189 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.081133 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.081712 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.081720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.082063 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.082536 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.082549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.083395 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.083399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.084296 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.084306 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.084888 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.084892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.085097 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.085734 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.085752 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.087198 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.090044 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.090049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.090062 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.090322 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.090670 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.090673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.090674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.090812 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.091301 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.091306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.091556 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.091725 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.091992 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":157,"threads_started":1}
I20260809 07:53:05.094599 10827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.18.71.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab.
Entry for principal kudu/127.18.71.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab.
I20260809 07:53:05.103562 18717 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
/tmp/dist-test-taskxTNb9d/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:46275
--webserver_interface=127.18.71.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab
--principal=kudu/127.18.71.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.18.71.84:42947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.18.71.126:37381,127.18.71.125:37991,127.18.71.124:46275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 07:53:05.103878 10746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } attempt: 3
I20260809 07:53:05.104167 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.095167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51536)
0809 07:53:05.095295 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.095310 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.095387 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.095996 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.095998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.096135 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.096232 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.096237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.097349 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.097352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.097894 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.097901 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.098543 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.098545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.098628 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.099110 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.099124 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.100266 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.102348 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.102351 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.102352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.102535 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.102934 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.102937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.102939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.103043 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.103509 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.103513 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.103627 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.103748 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.103920 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 07:53:05.104862 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.095278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51536 (local address 127.18.71.125:37991)
0809 07:53:05.095611 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.095621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.095642 (+ 21us) server_negotiation.cc:407] Connection header received
0809 07:53:05.095680 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.095685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.095836 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.096023 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.096331 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.097242 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.097992 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.098443 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.098528 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.100389 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.100433 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.100450 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.100511 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.102248 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.102688 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.102697 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.102702 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.102804 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.103154 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.103156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.103157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.103397 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.103504 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.104401 (+ 897us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.104599 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":144,"threads_started":1}
I20260809 07:53:05.107409 10827 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:05.108160 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.107885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39224)
0809 07:53:05.108006 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.108090 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 07:53:05.108393 10770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
I20260809 07:53:05.109376 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.109138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39228)
0809 07:53:05.109274 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.109322 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
W20260809 07:53:05.110062 10827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46275: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
I20260809 07:53:05.113818 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.104796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51538 (local address 127.18.71.125:37991)
0809 07:53:05.105091 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.105097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.105113 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:53:05.105150 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.105156 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.105207 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.105297 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.105774 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.106569 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.107620 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.107947 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.108047 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.110605 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.110609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.110612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.110634 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.112075 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.112695 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.112697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.112699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.112768 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.113076 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.113077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.113078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.113217 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.113285 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.113634 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.113722 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":176,"threads_started":1}
I20260809 07:53:05.113977 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.104295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51538)
0809 07:53:05.104572 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.104596 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.104670 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.105347 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.105357 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.105505 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.105656 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.105665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.106708 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.106713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.107478 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.107491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.108069 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.108072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.108201 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.108681 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.108693 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.110428 (+ 1735us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.112195 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.112201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.112206 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.112583 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.112868 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.112873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.112877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.112987 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.113314 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.113318 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.113444 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.113565 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.113742 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260809 07:53:05.115013 10746 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:05.115688 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.115472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39244)
0809 07:53:05.115582 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.115634 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 07:53:05.116346 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.116116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39260)
0809 07:53:05.116259 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.116299 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
W20260809 07:53:05.116518 10746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46275: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
I20260809 07:53:05.147682 10746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } attempt: 1
I20260809 07:53:05.148635 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.148288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39266)
0809 07:53:05.148439 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.148547 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 07:53:05.149366 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.149166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39270)
0809 07:53:05.149274 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.149317 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 07:53:05.149533 10746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46275: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
I20260809 07:53:05.159402 10827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } attempt: 1
I20260809 07:53:05.160205 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.159941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39276)
0809 07:53:05.160077 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.160137 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 07:53:05.160943 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.160734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39290)
0809 07:53:05.160841 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.160892 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 07:53:05.161067 10827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46275: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
W20260809 07:53:05.223802 10834 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 07:53:05.224148 10834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 07:53:05.224201 10834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 07:53:05.224275 10834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 07:53:05.224679 10746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } attempt: 2
I20260809 07:53:05.225550 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.225241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39298)
0809 07:53:05.225397 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.225465 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 07:53:05.226241 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.226036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39312)
0809 07:53:05.226139 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.226190 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 07:53:05.226358 10746 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46275: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
W20260809 07:53:05.228884 10834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 07:53:05.228960 10834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 07:53:05.229009 10834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 07:53:05.229063 10834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 07:53:05.229099 10834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 07:53:05.229224 10827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } attempt: 2
I20260809 07:53:05.229885 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.229608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39316)
0809 07:53:05.229761 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.229821 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 07:53:05.230579 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.230354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39330)
0809 07:53:05.230468 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.230516 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 07:53:05.230708 10827 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.18.71.124:46275: Network error: Client connection negotiation failed: client connection to 127.18.71.124:46275: connect: Connection refused (error 111)
I20260809 07:53:05.234929 10834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.18.71.84:42947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.18.71.126:37381,127.18.71.125:37991,127.18.71.124:46275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/krb5kdc/kudu_127.18.71.124.keytab
--principal=kudu/127.18.71.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.18.71.124:46275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.18.71.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 07:01:58 UTC on ccycloud.quasar-fnohdt.root.comops.site
build id 4575851
I20260809 07:53:05.236040 10834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 07:53:05.237179 10834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 07:53:05.245826 10841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:05.245855 10842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 07:53:05.245832 10845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 07:53:05.245941 10834 server_base.cc:1034] running on GCE node
I20260809 07:53:05.246592 10834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 07:53:05.247682 10834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 07:53:05.248842 10834 hybrid_clock.cc:648] HybridClock initialized: now 1786261985248824 us; error 31 us; skew 500 ppm
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 07:53:05.251597 10834 init.cc:377] Logged in from keytab as kudu/127.18.71.124@KRBTEST.COM (short username kudu)
I20260809 07:53:05.252569 10834 webserver.cc:460] Webserver started at http://127.18.71.124:39635/ using document root <none> and password file <none>
I20260809 07:53:05.253175 10834 fs_manager.cc:359] Metadata directory not provided
I20260809 07:53:05.253238 10834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 07:53:05.253473 10834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 07:53:05.255538 10834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data/instance:
uuid: "7cdcb66d122548a4befb2bd0e3bb6f72"
format_stamp: "Formatted at 2026-08-09 07:53:05 on dist-test-slave-10pc"
server_key: "79dcf35b97951557d207e510d42cdd90"
server_key_iv: "e252f40987e50b62ba45235406b5d28c"
server_key_version: "encryptionkey@0"
I20260809 07:53:05.256129 10834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/wal/instance:
uuid: "7cdcb66d122548a4befb2bd0e3bb6f72"
format_stamp: "Formatted at 2026-08-09 07:53:05 on dist-test-slave-10pc"
server_key: "79dcf35b97951557d207e510d42cdd90"
server_key_iv: "e252f40987e50b62ba45235406b5d28c"
server_key_version: "encryptionkey@0"
I20260809 07:53:05.259810 10834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 07:53:05.262267 10853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 07:53:05.263473 10834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 07:53:05.263602 10834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data,/tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/wal
uuid: "7cdcb66d122548a4befb2bd0e3bb6f72"
format_stamp: "Formatted at 2026-08-09 07:53:05 on dist-test-slave-10pc"
server_key: "79dcf35b97951557d207e510d42cdd90"
server_key_iv: "e252f40987e50b62ba45235406b5d28c"
server_key_version: "encryptionkey@0"
I20260809 07:53:05.263710 10834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 07:53:05.286012 10834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 07:53:05.289024 10834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 07:53:05.289245 10834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 07:53:05.296571 10834 rpc_server.cc:307] RPC server started. Bound to: 127.18.71.124:46275
I20260809 07:53:05.296888 10910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.18.71.124:46275 every 8 connection(s)
I20260809 07:53:05.297613 10834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/data/info.pb
I20260809 07:53:05.300520 10911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 07:53:05.303661 18717 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu as pid 10834
I20260809 07:53:05.303823 18717 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxTNb9d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786261695308648-18717-0/minicluster-data/master-2/wal/instance
I20260809 07:53:05.304102 18717 external_mini_cluster.cc:1428] Setting key 53f6d971bdbf3f7df82dcf3afe06f7ba
I20260809 07:53:05.304601 10911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:53:05.316839 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.306569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.18.71.126:37381)
0809 07:53:05.306724 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.306729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.306978 (+ 249us) server_negotiation.cc:407] Connection header received
0809 07:53:05.307354 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.307358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.307414 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.307513 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.308333 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.308930 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.309925 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.310159 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.310265 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.312987 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.312991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.312993 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.313017 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.314726 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.315282 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.315286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.315287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.315384 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.315708 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.315713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.315715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.315921 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.316029 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.316545 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.316664 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 07:53:05.317764 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.306465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.126:37381 (local address 127.0.0.1:54826)
0809 07:53:05.306843 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.306883 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.307166 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.307581 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.307587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.307944 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.308190 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.308202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.309041 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.309047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.309802 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.309811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.310316 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.310320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.310471 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.311083 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.311109 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.312797 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.314835 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.314843 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.314850 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.315183 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.315464 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.315467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.315469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.315609 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.316030 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.316047 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.316286 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.316490 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.316803 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260809 07:53:05.317767 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.305568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39342 (local address 127.18.71.124:46275)
0809 07:53:05.305997 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.306017 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.306052 (+ 35us) server_negotiation.cc:407] Connection header received
0809 07:53:05.306954 (+ 902us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.306976 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.307279 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.307469 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.308336 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.309251 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.310111 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.310483 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.310598 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.313400 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.313405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.313415 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.313458 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.315273 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.315673 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.315679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.315681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.315764 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.316058 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.316061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.316062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.316334 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.316564 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.316821 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.316960 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"spinlock_wait_cycles":896,"thread_start_us":137,"threads_started":1}
I20260809 07:53:05.319388 10911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:05.321861 10827 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } attempt: 3
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
I20260809 07:53:05.325493 10746 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } attempt: 3
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.126@KRBTEST.COM
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.125@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:53:05.329721 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.320024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51542 (local address 127.18.71.125:37991)
0809 07:53:05.320172 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.320194 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.320207 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:53:05.320280 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.320284 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.320335 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.320415 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.321010 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.321707 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.322702 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.322916 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.323019 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.325528 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.325531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.325534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.325558 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.327512 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.327990 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.327993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.327995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.328076 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.328446 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.328469 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.328471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.328690 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.328784 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.329414 (+ 630us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.329536 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":57}
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.126@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:53:05.331554 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.319891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.125:37991 (local address 127.0.0.1:51542)
0809 07:53:05.320032 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.320046 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.320141 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.320526 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.320530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.320745 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.320877 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.320884 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.321822 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.321825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.322590 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.322598 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.323029 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.323033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.323151 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.323894 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.323910 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.325391 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.327636 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.327639 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.327641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.327878 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.328220 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.328223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.328225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.328318 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.329125 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.329129 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.329218 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.331205 (+ 1987us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.331349 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 07:53:05.332607 10911 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } has no permanent_uuid. Determining permanent_uuid...
I20260809 07:53:05.332958 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.322233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39356)
0809 07:53:05.322378 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.322391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.322458 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.322796 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.322799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.322977 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.323117 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.323124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.324019 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.324024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.324682 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.324693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.325406 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.325410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.325527 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.326244 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.326264 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.327837 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.331039 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.331043 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.331045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.331303 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.331601 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.331605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.331607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.331710 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.332356 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.332362 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.332498 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.332608 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.332762 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":3}
I20260809 07:53:05.333155 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.320401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54840 (local address 127.18.71.126:37381)
0809 07:53:05.320521 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.320525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.320537 (+ 12us) server_negotiation.cc:407] Connection header received
0809 07:53:05.320655 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.320658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.320703 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.320787 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.321454 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.322065 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.323102 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.323431 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.323531 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.328794 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.328798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.328800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.328823 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.330682 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.331196 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.331201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.331206 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.331303 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.331671 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.331675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.331677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.331929 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.332152 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.332863 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.332991 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 07:53:05.333669 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.322487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39356 (local address 127.18.71.124:46275)
0809 07:53:05.322577 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.322581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.322592 (+ 11us) server_negotiation.cc:407] Connection header received
0809 07:53:05.322635 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.322637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.322671 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.323244 (+ 573us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.323322 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.323906 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.324825 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.325274 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.325389 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.327975 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.327979 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.327981 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.328008 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.330917 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.331418 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.331421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.331423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.331495 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.331986 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.331989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.331991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.332211 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.332305 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.333372 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.333479 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 07:53:05.336380 10749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.325917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39364)
0809 07:53:05.326025 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.326039 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.326118 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.326624 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.326628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.326827 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.326945 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.326954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.328127 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.328131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.328995 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.329008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.329619 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.329623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.329701 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.330155 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.330172 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.332068 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.334775 (+ 2707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.334778 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.334780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.335018 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.335339 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.335342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.335344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.335446 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.335877 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.335880 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.335971 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.336075 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.336206 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 07:53:05.336472 10916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.325952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.18.71.124:46275)
0809 07:53:05.326341 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.326354 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.326369 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:53:05.326408 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.326413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.326451 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.326593 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.327052 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.327981 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.329151 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.329485 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.329929 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.332204 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.332208 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.332210 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.332230 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.334653 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.335145 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.335149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.335152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.335239 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.335550 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.335553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.335555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.335770 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.335865 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.336158 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.336269 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":160,"threads_started":1}
I20260809 07:53:05.337584 10827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: Bootstrap starting.
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261985, etypes {rep=17 tkt=17 ses=17}, kudu/127.18.71.124@KRBTEST.COM for kudu/127.18.71.124@KRBTEST.COM
I20260809 07:53:05.339475 10746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1: Bootstrap starting.
I20260809 07:53:05.340749 10827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:05.341718 10827 log.cc:824] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:05.341758 10746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:05.342583 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.333183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39366 (local address 127.18.71.124:46275)
0809 07:53:05.334011 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.334016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.334032 (+ 16us) server_negotiation.cc:407] Connection header received
0809 07:53:05.334069 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.334075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.334130 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.334219 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.334717 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.335403 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.336309 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.336677 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.336749 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.339019 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.339022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.339025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.339053 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.340841 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.341323 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.341326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.341327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.341403 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.341684 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.341687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.341690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.341907 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.341987 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.342325 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.342430 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":118,"threads_started":1}
I20260809 07:53:05.342763 10746 log.cc:824] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:05.343025 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.333124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.18.71.124:46275 (local address 127.0.0.1:39366)
0809 07:53:05.333407 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 07:53:05.333420 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 07:53:05.333487 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 07:53:05.334251 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 07:53:05.334253 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 07:53:05.334478 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 07:53:05.334585 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.334604 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.335521 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.335524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.336198 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 07:53:05.336208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.336869 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.336872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 07:53:05.336945 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.337389 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 07:53:05.337404 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 07:53:05.338872 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 07:53:05.340958 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.340961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.340963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.341215 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.341497 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 07:53:05.341500 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 07:53:05.341501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 07:53:05.341586 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 07:53:05.342050 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 07:53:05.342054 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 07:53:05.342157 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 07:53:05.342751 (+ 594us) client_negotiation.cc:240] Negotiation successful
0809 07:53:05.342883 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228}
I20260809 07:53:05.344007 10827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: No bootstrap required, opened a new log
I20260809 07:53:05.345041 10746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1: No bootstrap required, opened a new log
I20260809 07:53:05.345920 10911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: Bootstrap starting.
Aug 09 07:53:05 dist-test-slave-10pc krb5kdc[10660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786261984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.18.71.125@KRBTEST.COM
I20260809 07:53:05.348372 10911 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: Neither blocks nor log segments found. Creating new log.
I20260809 07:53:05.348608 10827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.348874 10827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d325f792da240328024ec9d31384508, State: Initialized, Role: FOLLOWER
I20260809 07:53:05.348932 10746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.349146 10746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06634d9dfb4c406a8ff032d454152ca1, State: Initialized, Role: FOLLOWER
I20260809 07:53:05.349227 10911 log.cc:824] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: Log is configured to *not* fsync() on all Append() calls
I20260809 07:53:05.349246 10827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.349499 10746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.350095 10922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.350220 10922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:05.350740 10827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:05.350807 10746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:05.352530 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.340609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51544 (local address 127.18.71.125:37991)
0809 07:53:05.341078 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.341082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.341095 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:53:05.341131 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.341135 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.341174 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.341246 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.342395 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.343132 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.344632 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.344930 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.345024 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.348193 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.348197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.348200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.348225 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.350248 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.350948 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.350951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.350953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.351036 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.351368 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.351372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.351374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.351605 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.351798 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.352193 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.352304 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389}
W20260809 07:53:05.355816 10946 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:05.355816 10947 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:05.355893 10946 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 07:53:05.355893 10947 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:05.356266 10920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.356393 10920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:53:05.356696 10695 tablet.cc:2369] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 07:53:05.358042 18717 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 07:53:05.358932 10911 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: No bootstrap required, opened a new log
I20260809 07:53:05.359593 10720 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "06634d9dfb4c406a8ff032d454152ca1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54840
I20260809 07:53:05.359797 10720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 07:53:05.359886 10720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:05.361315 10720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.362548 10800 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06634d9dfb4c406a8ff032d454152ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3d325f792da240328024ec9d31384508"
I20260809 07:53:05.362756 10800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:05.363131 10911 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.363369 10911 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cdcb66d122548a4befb2bd0e3bb6f72, State: Initialized, Role: FOLLOWER
I20260809 07:53:05.363690 10911 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.364090 10800 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06634d9dfb4c406a8ff032d454152ca1 in term 1.
I20260809 07:53:05.361016 10673 master_runner.cc:186] Error getting master registration for 127.18.71.124:46275: OK, instance_id { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" instance_seqno: 1786261985294601 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:05.364444 10673 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:05.364534 10689 leader_election.cc:302] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06634d9dfb4c406a8ff032d454152ca1, 3d325f792da240328024ec9d31384508; no voters:
I20260809 07:53:05.364840 10922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 07:53:05.365141 10922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 06634d9dfb4c406a8ff032d454152ca1, State: Running, Role: LEADER
I20260809 07:53:05.365182 10911 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 07:53:05.365505 10922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } }
I20260809 07:53:05.367221 10922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06634d9dfb4c406a8ff032d454152ca1. Latest consensus state: current_term: 1 leader_uuid: "06634d9dfb4c406a8ff032d454152ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.367350 10922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:05.367748 10720 leader_election.cc:288] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [CANDIDATE]: Term 1 election: Requested vote from peers 3d325f792da240328024ec9d31384508 (127.18.71.125:37991), 7cdcb66d122548a4befb2bd0e3bb6f72 (127.18.71.124:46275)
I20260809 07:53:05.368721 10885 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "06634d9dfb4c406a8ff032d454152ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72"
I20260809 07:53:05.368958 10885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 07:53:05.369911 10966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 07:53:05.370307 10885 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06634d9dfb4c406a8ff032d454152ca1 in term 1.
I20260809 07:53:05.371048 10966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 07:53:05.359495 10753 master_runner.cc:186] Error getting master registration for 127.18.71.124:46275: OK, instance_id { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" instance_seqno: 1786261985294601 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 07:53:05.372665 10753 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 07:53:05.372645 10950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.372787 10950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 07:53:05.376123 10965 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 07:53:05.376204 10965 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 07:53:05.378404 10885 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [term 1 FOLLOWER]: Refusing update from remote peer 06634d9dfb4c406a8ff032d454152ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:05.379041 10951 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:53:05.380108 10800 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [term 1 FOLLOWER]: Refusing update from remote peer 06634d9dfb4c406a8ff032d454152ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 07:53:05.381006 10951 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 07:53:05.384771 10950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06634d9dfb4c406a8ff032d454152ca1. Latest consensus state: current_term: 1 leader_uuid: "06634d9dfb4c406a8ff032d454152ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.384891 10950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:05.387398 10920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06634d9dfb4c406a8ff032d454152ca1. Latest consensus state: current_term: 1 leader_uuid: "06634d9dfb4c406a8ff032d454152ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.387538 10920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:05.389086 10951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06634d9dfb4c406a8ff032d454152ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.389240 10951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:05.390410 10950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06634d9dfb4c406a8ff032d454152ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.390515 10950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:05.390909 10977 mvcc.cc:205] Tried to move back new op lower bound from 7316529092103077888 to 7316529092060315648. Current Snapshot: MvccSnapshot[applied={T|T < 7316529092103077888}]
I20260809 07:53:05.391089 10971 mvcc.cc:205] Tried to move back new op lower bound from 7316529092103077888 to 7316529092060315648. Current Snapshot: MvccSnapshot[applied={T|T < 7316529092103077888}]
I20260809 07:53:05.393378 10966 catalog_manager.cc:1347] Generated new cluster ID: dfdd831bab08419eb5989f5174affe6a
I20260809 07:53:05.393447 10966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 07:53:05.397660 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.378823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39372 (local address 127.18.71.124:46275)
0809 07:53:05.378926 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.378930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.388244 (+ 9314us) server_negotiation.cc:407] Connection header received
0809 07:53:05.388429 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.388432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.388501 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.388553 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.389829 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.390539 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.391891 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.392192 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.392268 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.394294 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.394297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.394300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.394326 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.395954 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.396402 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.396405 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.396407 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.396507 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.396788 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.396791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.396792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.396980 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.397068 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.397394 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.397502 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 07:53:05.402957 10922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "06634d9dfb4c406a8ff032d454152ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.403095 10922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1 [sys.catalog]: This master's current role is: LEADER
I20260809 07:53:05.404050 10920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "06634d9dfb4c406a8ff032d454152ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "06634d9dfb4c406a8ff032d454152ca1" member_type: VOTER last_known_addr { host: "127.18.71.126" port: 37381 } } peers { permanent_uuid: "3d325f792da240328024ec9d31384508" member_type: VOTER last_known_addr { host: "127.18.71.125" port: 37991 } } peers { permanent_uuid: "7cdcb66d122548a4befb2bd0e3bb6f72" member_type: VOTER last_known_addr { host: "127.18.71.124" port: 46275 } } }
I20260809 07:53:05.404204 10920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 07:53:05.405038 10979 mvcc.cc:205] Tried to move back new op lower bound from 7316529092103077888 to 7316529092060315648. Current Snapshot: MvccSnapshot[applied={T|T < 7316529092103077888}]
I20260809 07:53:05.411549 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.377648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51556 (local address 127.18.71.125:37991)
0809 07:53:05.377757 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.377761 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.387912 (+ 10151us) server_negotiation.cc:407] Connection header received
0809 07:53:05.388070 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.388074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.388132 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.388212 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.397366 (+ 9154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.398226 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.400622 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.401068 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.401154 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.406720 (+ 5566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.406724 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.406727 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.406758 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.409185 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.409709 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.409713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.409715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.409805 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.410227 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.410231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.410233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.410463 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.410597 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.411246 (+ 649us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.411377 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 07:53:05.414698 10750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:05.374625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54842 (local address 127.18.71.126:37381)
0809 07:53:05.374737 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.374742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.380802 (+ 6060us) server_negotiation.cc:407] Connection header received
0809 07:53:05.380998 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.381001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.381051 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.381105 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.388318 (+ 7213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.389255 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.399542 (+ 10287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:05.399932 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:05.400037 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:05.405694 (+ 5657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 07:53:05.405699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 07:53:05.405702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 07:53:05.405733 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 07:53:05.407771 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.413148 (+ 5377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.413152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.413154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.413255 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 07:53:05.413653 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 07:53:05.413657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 07:53:05.413659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 07:53:05.413887 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 07:53:05.414045 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:05.414406 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 07:53:05.414535 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 07:53:05.433738 10966 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 07:53:05.434999 10966 catalog_manager.cc:1504] Loading token signing keys...
I20260809 07:53:05.449904 10966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06634d9dfb4c406a8ff032d454152ca1: Generated new TSK 0
I20260809 07:53:05.450941 10966 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 07:53:05.487840 10750 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.484345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54848 (local address 127.18.71.126:37381)
0809 07:53:05.484510 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.484515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.486594 (+ 2079us) server_negotiation.cc:407] Connection header received
0809 07:53:05.486817 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.486821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.486876 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.486935 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.487677 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54848: BlockingRecv error: recv got EOF from 127.0.0.1:54848 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 07:53:05.489593 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.485344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.18.71.125:37991)
0809 07:53:05.485488 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.485493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.488555 (+ 3062us) server_negotiation.cc:407] Connection header received
0809 07:53:05.488744 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.488749 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.488812 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.488873 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.489451 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51562: BlockingRecv error: recv got EOF from 127.0.0.1:51562 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 07:53:05.490844 10914 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.486031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.18.71.124:46275)
0809 07:53:05.486152 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.486158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.489014 (+ 2856us) server_negotiation.cc:407] Connection header received
0809 07:53:05.490004 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.490008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.490069 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.490134 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.490713 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39388: BlockingRecv error: recv got EOF from 127.0.0.1:39388 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 07:53:05.495132 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.493553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51574 (local address 127.18.71.125:37991)
0809 07:53:05.493647 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.493651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.494056 (+ 405us) server_negotiation.cc:407] Connection header received
0809 07:53:05.494284 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.494287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.494390 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.494444 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.495038 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51574: BlockingRecv error: recv got EOF from 127.0.0.1:51574 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 07:53:05.500304 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.498870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51586 (local address 127.18.71.125:37991)
0809 07:53:05.498967 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.498971 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.499429 (+ 458us) server_negotiation.cc:407] Connection header received
0809 07:53:05.499615 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.499618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.499680 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.499734 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.500209 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51586: BlockingRecv error: recv got EOF from 127.0.0.1:51586 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 07:53:05.506321 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.504811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51602 (local address 127.18.71.125:37991)
0809 07:53:05.504908 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.504912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.505434 (+ 522us) server_negotiation.cc:407] Connection header received
0809 07:53:05.505627 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.505630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.505678 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.505730 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.506230 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51602: BlockingRecv error: recv got EOF from 127.0.0.1:51602 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 07:53:05.514276 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.512671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.18.71.125:37991)
0809 07:53:05.512810 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.512814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.513329 (+ 515us) server_negotiation.cc:407] Connection header received
0809 07:53:05.513542 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.513546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.513593 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.513644 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.514181 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51618: BlockingRecv error: recv got EOF from 127.0.0.1:51618 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 07:53:05.526285 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.524662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.18.71.125:37991)
0809 07:53:05.524775 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.524779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.525306 (+ 527us) server_negotiation.cc:407] Connection header received
0809 07:53:05.525518 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.525522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.525572 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.525630 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.526180 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51622: BlockingRecv error: recv got EOF from 127.0.0.1:51622 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 07:53:05.547523 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.545513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.18.71.125:37991)
0809 07:53:05.545651 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.545656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.546312 (+ 656us) server_negotiation.cc:407] Connection header received
0809 07:53:05.546565 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.546571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.546647 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.546716 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.547394 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51628: BlockingRecv error: recv got EOF from 127.0.0.1:51628 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 07:53:05.584633 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.582627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51636 (local address 127.18.71.125:37991)
0809 07:53:05.582768 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.582773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.583394 (+ 621us) server_negotiation.cc:407] Connection header received
0809 07:53:05.583626 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.583631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.583694 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.583752 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.584438 (+ 686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51636: BlockingRecv error: recv got EOF from 127.0.0.1:51636 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 07:53:05.652930 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.651753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.18.71.125:37991)
0809 07:53:05.651887 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.651893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.652067 (+ 174us) server_negotiation.cc:407] Connection header received
0809 07:53:05.652149 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.652152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.652248 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.652347 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.652830 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51640: BlockingRecv error: recv got EOF from 127.0.0.1:51640 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 07:53:05.784243 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:05.783083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51644 (local address 127.18.71.125:37991)
0809 07:53:05.783231 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:05.783235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:05.783416 (+ 181us) server_negotiation.cc:407] Connection header received
0809 07:53:05.783490 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:05.783494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:05.783548 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:05.783666 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:05.784140 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51644: BlockingRecv error: recv got EOF from 127.0.0.1:51644 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 07:53:06.043989 10832 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 07:53:06.042699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.18.71.125:37991)
0809 07:53:06.042862 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:06.042866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:06.042974 (+ 108us) server_negotiation.cc:407] Connection header received
0809 07:53:06.043181 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:06.043184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:06.043233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:06.043325 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 07:53:06.043882 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51652: BlockingRecv error: recv got EOF from 127.0.0.1:51652 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 07:53:06.357201 10947 catalog_manager.cc:1259] Loaded cluster ID: dfdd831bab08419eb5989f5174affe6a
I20260809 07:53:06.357311 10947 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: loading cluster ID for follower catalog manager: success
I20260809 07:53:06.358994 10947 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: acquiring CA information for follower catalog manager: success
I20260809 07:53:06.360168 10947 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3d325f792da240328024ec9d31384508: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:53:06.377600 10965 catalog_manager.cc:1259] Loaded cluster ID: dfdd831bab08419eb5989f5174affe6a
I20260809 07:53:06.377697 10965 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: loading cluster ID for follower catalog manager: success
I20260809 07:53:06.379294 10965 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: acquiring CA information for follower catalog manager: success
I20260809 07:53:06.380409 10965 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7cdcb66d122548a4befb2bd0e3bb6f72: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 07:53:06.562064 11056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:06.558375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.18.71.125:37991)
0809 07:53:06.558609 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:06.558618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:06.558706 (+ 88us) server_negotiation.cc:407] Connection header received
0809 07:53:06.558805 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:06.558808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:06.558952 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:06.559063 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:06.559354 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.559877 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.560692 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.560909 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.560987 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:06.561283 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:06.561736 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:06.561829 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:06.561882 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 07:53:06.561941 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
I20260809 07:53:06.568881 11062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:06.565166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34266 (local address 127.18.71.124:46275)
0809 07:53:06.565382 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:06.565386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:06.565399 (+ 13us) server_negotiation.cc:407] Connection header received
0809 07:53:06.565499 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:06.565501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:06.565617 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:06.565719 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:06.565979 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.566549 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.567410 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.567678 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.567759 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:06.568043 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:06.568564 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:06.568660 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:06.568715 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 07:53:06.568761 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260809 07:53:06.575800 11062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:06.572048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34282 (local address 127.18.71.124:46275)
0809 07:53:06.572184 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:06.572188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:06.572390 (+ 202us) server_negotiation.cc:407] Connection header received
0809 07:53:06.572435 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:06.572437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:06.572523 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:06.572602 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:06.572917 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.573400 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.574525 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.574794 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.574889 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:06.575164 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:06.575238 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:06.575305 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:06.575613 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 07:53:06.575669 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 07:53:06.575915 11056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:06.571597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55220 (local address 127.18.71.125:37991)
0809 07:53:06.571723 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:06.571727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:06.572273 (+ 546us) server_negotiation.cc:407] Connection header received
0809 07:53:06.572403 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:06.572406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:06.572486 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:06.572562 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:06.572914 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.573677 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.574725 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.575024 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.575161 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:06.575440 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:06.575513 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:06.575584 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:06.575731 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 07:53:06.575780 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 07:53:06.577515 11069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 07:53:06.571601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53980 (local address 127.18.71.126:37381)
0809 07:53:06.571853 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 07:53:06.571858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 07:53:06.571873 (+ 15us) server_negotiation.cc:407] Connection header received
0809 07:53:06.572064 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 07:53:06.572067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 07:53:06.572270 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 07:53:06.572411 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 07:53:06.572821 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.573677 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.575660 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 07:53:06.576252 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 07:53:06.576364 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 07:53:06.576845 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 07:53:06.577109 (+ 264us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 07:53:06.577183 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 07:53:06.577260 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 07:53:06.577312 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":65,"threads_started":1}
I20260809 07:53:06.588327 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10673
I20260809 07:53:06.598119 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10753
I20260809 07:53:06.606547 18717 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxTNb9d/build/debug/bin/kudu with pid 10834
2026-08-09T07:53:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3139 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6265 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291261 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST